Advertising (This ad goes away for registered users. You can Login or Register)

memory management: FPL...

This is the development forum of the half-byte loader project. For general Half Byte Loader questions, visit the Half Byte Loader forum.
Forum rules
This forum is for HBL Development discussions ONLY. For User support or HBL general discussions, go to viewforum.php?f=3 . Messages that are not development related will be deleted.
wololo
Site Admin
Posts: 3621
Joined: Wed Oct 15, 2008 12:42 am
Location: Japan

memory management: FPL...

Post by wololo »

I'm running into this issue with a specific game:
The game is allocating memory using

sceKernelCreateFpl
Followed by
sceKernelTryAllocateFpl

The game however does not import
sceKernelFreeFpl
or
sceKernelDeleteFpl

Since in 6.60 syscall estimation seems broken, I can't use any of those two delete functions... so I'm basically losing about 20MB because of that memory pool...
any idea on how to patch HBL efficiently on this one?
Advertising
If you need US PSN Codes, this technique is what I recommend.

Looking for guest bloggers and news hunters here at wololo.net, PM me!
JJS
Big Beholder
Posts: 1416
Joined: Mon Sep 27, 2010 2:18 pm
Contact:

Re: memory management: FPL...

Post by JJS »

You might be able to get sceKernelFreeFpl from P5 if you can load http://moonlight.lan.st/3.0x/vsh/module ... ogimp.html .

E: Actually, loading that module is what p5_copy_stubs_savedata() does.

E2: Since the syscall will become unavailable after the dialog closes, I guess you have to do the freeing while the dialog is open. So the P5 opening might have to be performed in the memory cleaning stage of HBL instead of in the binary loader.

E3: Better idea! There are utility (user mode) modules that import valuable functions. E.g. sceKernelFreeFpl is in PSP_MODULE_NET_SSL. It depends on PSP_MODULE_NET_COMMON and PSP_MODULE_NET_INET. Loading and unloading these is more reliable than the savedata stuff.
Also thread management functions are imported in PSP_MODULE_NET_COMMON, so it might be interesting to have that module always loaded to work around that other thread function problem we have.
Advertising
wololo
Site Admin
Posts: 3621
Joined: Wed Oct 15, 2008 12:42 am
Location: Japan

Re: memory management: FPL...

Post by wololo »

Ok, thanks for the suggestions.
That one is a tough nut, we really don't have much memory to work with... (less than 50kB initially... trying to be clever with where to load HBL, but not super helpful)

Also, sceUtilityGetSystemParamInt is not imported, which prevents me from calling these 2 things in P5 stubs:

sceUtilityGetSystemParamInt(PSP_SYSTEMPARAM_ID_INT_LANGUAGE, &dialog.base.language); // Prompt language
sceUtilityGetSystemParamInt(PSP_SYSTEMPARAM_ID_INT_UNKNOWN, &dialog.base.buttonSwap); // X/O button swap

Do you know if those are really useful ?
If you need US PSN Codes, this technique is what I recommend.

Looking for guest bloggers and news hunters here at wololo.net, PM me!
JJS
Big Beholder
Posts: 1416
Joined: Mon Sep 27, 2010 2:18 pm
Contact:

Re: memory management: FPL...

Post by JJS »

You can use whatever value you want for them, just both times 0 is fine I think (you might want to check the language constants if 0 causes it to fail). The auto save dialog is not shown anyway, so the values don't really matter.
wololo
Site Admin
Posts: 3621
Joined: Wed Oct 15, 2008 12:42 am
Location: Japan

Re: memory management: FPL...

Post by wololo »

Right now the game is freezing on my when running the p5 stubs functions...
but I'm not surprising, it's been freezing all the time for no reason, I'm assuming that with less 50kB of ram free in p2, any action can possibly kill the console.

If my understanding is correct, the good point of loading that specific module is that it uses p5 instead of p2. I am assuming other modules might be using p2 instead, which is too crowded right now...
I need to add a few debug statements here and there to see exactly what is failing and where.

I'll try to keep you updated, it's quite late here so I'll go to bed for now, was not expecting to spend so much time on this. On a brighter side, I improved my gen_exploit_config.rb file... although I'm realizing it is not leveraging the fact that some functions can be defined outside of the game's imports... room for improvment here I guess... probably that ruby script could also be doing some stubs search on its own...
If you need US PSN Codes, this technique is what I recommend.

Looking for guest bloggers and news hunters here at wololo.net, PM me!
wololo
Site Admin
Posts: 3621
Joined: Wed Oct 15, 2008 12:42 am
Location: Japan

Re: memory management: FPL...

Post by wololo »

JJS wrote: E3: Better idea! There are utility (user mode) modules that import valuable functions. E.g. sceKernelFreeFpl is in PSP_MODULE_NET_SSL. It depends on PSP_MODULE_NET_COMMON and PSP_MODULE_NET_INET. Loading and unloading these is more reliable than the savedata stuff.
Also thread management functions are imported in PSP_MODULE_NET_COMMON, so it might be interesting to have that module always loaded to work around that other thread function problem we have.
I think we should strongly investigate this possibility. I am now running in several cases of games (especially old games) missing some essential functions for HBL to work.
Especially threading functions (the ones that take a threadid as a parameter), but also (sadly) new sceUtilityLoad/Unload Module (these games usually have access to sceKernelLoadModule only).

Your workaround that kills the thread is unfortunately not enough because at some point HBL also has to kill some running threads, and that becomes difficult.

I haven't had enough time recently to look into that, but basically games that do not import either sceKernelTerminateDeleteThread or sceKernelTerminateThread+sceKernelDeleteThread are for the moment impossible to port to VHBL.
Same issue with games that do not import sceUtilityLoad/Unload

So, loading modules to import additional functions, (or figuring out "defnitely" why syscall estimation doesn't work anymore ?) seems like a good way to go now.

I wish some "new" people could look into that, as it feels to me now that all the "major" contributors to HBL are busy and/or moved on.
If you need US PSN Codes, this technique is what I recommend.

Looking for guest bloggers and news hunters here at wololo.net, PM me!
wololo
Site Admin
Posts: 3621
Joined: Wed Oct 15, 2008 12:42 am
Location: Japan

Re: memory management: FPL...

Post by wololo »

Hi JJS, as we discussed, loading/unloading modules seems to help by importing more functions. The issue I'm running into right now is that syscall estimation is not using the "perfect syscalls" because I am not sure where/how to retrieve the values that are supposed to go in the "SYSCALL_OFFSETS_XXX" values... and it seems these values are different for CFW and OFW?

Alternatively, a possibility would be to find these syscalls from the loaded modules, somehow?
If you need US PSN Codes, this technique is what I recommend.

Looking for guest bloggers and news hunters here at wololo.net, PM me!
JJS
Big Beholder
Posts: 1416
Joined: Mon Sep 27, 2010 2:18 pm
Contact:

Re: memory management: FPL...

Post by JJS »

Perfect syscall estimation is not possible on > 6.20 because of the variable gap. Perfect syscalls rely on the fact that there is a specific order and offset between syscalls which does not hold true on the newer firmwares.

To find the syscalls from the utility modules, you could use the existing stub parsing function (build_nid_table). It only would have to be modified so that it doesn't use the hardcoded stub addresses for the exploited game and instead gets passed each address individually. The caller could then load a utility, determine the address of the stub in memory and pass that on. Adding the functions to the nid table should work without modification.

Of course there is the problem that syscall estimation for the HBL function takes place before cleaning the memory. This means that unloading utility modules (to load different ones) might very well crash the still running game.
wololo
Site Admin
Posts: 3621
Joined: Wed Oct 15, 2008 12:42 am
Location: Japan

Re: memory management: FPL...

Post by wololo »

Thanks.

I'm experimenting, I tried to load PSP_NET_COMMON, and can see the stubs in a user dump, the only question is how to find them at runtime... I'll see how fast it is to browse the ram for a library name, and if that can then be optimized (found one library name, other libraries must be close to that point...)

What's extremely frustrating is that everything I'm doing here is extremely easy, it's a dozen hours of work at most, but the problem is that I barely have 30 mins per day these days, just enough to "get in the zone" and not do anything useful.
Here's hoping some "new" guys with more free time than us will start spending the few hours it takes to help ;)

To summarize, here's the basic idea:
for (all user modules) {
sceUtilityLoadModule(modid);
searchStubs()
sceUtilityUnloadModule(modid);
}

This loop might have to happen twice:
-once in loader.c in order to "complete" HBL necessary functions (this is the most important step, this could get HBL to work on many more game exploits, especially the ones that have been causing trouble because the game is not importing enough functions)
- once in...err... tables.c ? to fill the nids table as much as possible and avoid syscall estimates
If you need US PSN Codes, this technique is what I recommend.

Looking for guest bloggers and news hunters here at wololo.net, PM me!
JJS
Big Beholder
Posts: 1416
Joined: Mon Sep 27, 2010 2:18 pm
Contact:

Re: memory management: FPL...

Post by JJS »

Well, we should not overcomplicate this. The real problem is not that the syscall estimation itself is broken but that the kernel refuses to resolve syscalls that are not used by any user mode module.

So the mere act of loading the utility modules gives us access to the functions that are imported by the modules. Parsing their import tables is merely icing on the cake.
Locked

Return to “Half Byte Loader Development”