AppGameKit Classic Chat->Bug - CreateFileFromMemblock
A
Arch-Ok Joined: 11th Jul 2019
Location: Bursa/T?RKIYE
Posted: 18th Nov 2021 15:47
I have created a server which periodacally saves registered user's data.

After about 8 hours of running, the server.exe stops with an error dialog box with literally no text, no data writing on it.

When I remove backup code it runs more than 8 hours, well I couldn't test more then 12 hours yet, but I know last line of my code before it produces that wired error with no error data.

It uses same memblock for gaining speed.

It fills the memblock with thw appopriate data which is 'bit precised'.

It then deletes the outdated file and immediately creates new user profile file.

At this point after deleting the file, it throws an error with empty dialog box, the file is deleted but the new one is not there.

So, createfilefrommemblock() fails internally.
P
PartTimeCoder Joined: 9th Mar 2015
Location: London UK
Posted: 18th Nov 2021 16:29
Yeah it sure sounds like an IMA.

if there is a memory corruption they maybe AppGameKit fails to "Format" the error message as it also uses memory for Unicode string conversion in its uString class, that would explain the empty dialog but it does not explain the initial corruption, are you 100% sure the buffer is large enough for the data and all offsets remain correct.

It sounds like you know exactly what you are doing here so you will understand what I am saying, createfilefrommemblock simply accesses the memory pointer and writes the data with fwrite() and immediately closes the file, along the way 2 error checks are performed, the 1st checks that the memblock exists and the 2nd checks that the file was created and is open to write, so it could also be a file lock issue and not memory, does anything else access the file?

Edit: But that still would not explain the empty dialog, my bet is on a memory overrun causing an IMA and curropting the AppGameKit heap allocation before it can format the error message, double check your offsets and maybe add some padding.
G
Game_Code_here Joined: 2nd Jun 2020
Location:
Posted: 18th Nov 2021 22:48
Yes this sounds like a memory problem.

What you can do is save on each hour to a new file then if the files are more then so many do whatever.

Could also be a firewall problem.

The empty data is caused by there being no saved file at the end, I'm sure, that is the only thing that makes sense.

But Just to let you know I have no experience with app game kits file write commands.

I'm just saying what makes sense.
A
Arch-Ok Joined: 11th Jul 2019
Location: Bursa/T?RKIYE
Posted: 19th Nov 2021 7:15
Thanks for the answers...

No memory leaks, no #gp fault exceptions...
The server has two real players and only 32 bots for now, and we can use it properly.
I open it at midnight and when I wake up I get this error, so no interactions made with the server during the night.

This morning it failed again.

I created a mini test;

+ Code Snippet
Do
i = CreateMemblock(1024 * 1024 * 64)
sync()
Loop


The error message;




I guess, AppGameKit pushes a structure data into a vector array for internal file handling and not popping it back from that array when using createfilefrommemblock??
type ftype
index as uint
path as tchar * 260
name as tchar * 260
size as ulongint
pos as ulongint
....
..
.

??

I repeat, the memblock is globally created once at the start and works for about 8 hours fine, so I'm pretty sure it is not the memblock fault.

This server must run like forever...

I've already created a winsock library in FreeBasic and I can switch to it which I don't want to do.

I can stop using createfilefrommemblock() and use writeinteger() but this will be so slow, yet I'll try this and see if it creates an error this night.
P
PartTimeCoder Joined: 9th Mar 2015
Location: London UK
Posted: 19th Nov 2021 8:27
No AppGameKit does not pop anything from the memblock with calling createfilefrommemblock, it leaves the buffer intact, AppGameKit does use an internal list to keep track of the memblocks but this merely holds the pointer address and gives you an unsigned int for ease of reference, the same list class is used everywhere in AppGameKit (for sprites, objects, text, edit etc etc) so I can say with certainly that this is not the problem or it would be bugging out in every project.

This:
+ Code Snippet
Do
i = CreateMemblock(1024 * 1024 * 64)
sync()
Loop


Is allocating 67MB per frame??, I would expect it to crash! lol

when freeing the memory, it does not crash
+ Code Snippet
Do
i = CreateMemblock(1024 * 1024 * 64)
DeleteMemblock(i)
sync()
Loop


The problem is not the memblock its the way its being used.

I am 99.99% sure this is an overrun issue causing an IMA (Invalid Memory Access)

try freeing the buffer once in a while and reallocating, also some things to bear in mind

the max buffer size AppGameKit will allow is < 100000000 bytes

agk allocates the memory as:
+ Code Snippet
pMem-&gt;m_pData = new unsigned char[ size ];


so if you have a signed char in your data you will get overrun? (as I understand it?)
nsigned char, which gives you at least the 0 to 255 range.


Edit: but I could be wrong on that, not great on memory stuff myself, I think the *sign* takes an extra byte or 2, maybe someone with more knowledge/experience can chime in here?
A
Arch-Ok Joined: 11th Jul 2019
Location: Bursa/T?RKIYE
Posted: 19th Nov 2021 11:24
Yes, excatly, the memblock list should have elements like pointer, size and index of an memblock. As I imagine a file structure just like that list. So, the pushing and popping is not from the memblock but an internal array which holds all the file data structures, but just a guess.

Yea, the test code allocates 64mb/frame to test what happens and what error code is given, the error code gives something at least.

The buffer is 1024 bytes only.

Yes, I am also thinking to deallocate the buffer and reallocate it after all player's datas are saved. (it backups 1 player data in 1 frames for speed.)

I am also thinking to end the server.exe after let's say 4 hours and restart it, so every 4 hours there will be some delays. To achive this, another exe will run the server.exe and if server.exe ends it will re-run the server.exe...

Nothing to do with the sign.

The sign takes only 1 bit.

Thanks.
P
PartTimeCoder Joined: 9th Mar 2015
Location: London UK
Posted: 19th Nov 2021 12:24
Well I hope you find a working solution, sorry I could not be more help, let us know how you get on.
A
Arch-Ok Joined: 11th Jul 2019
Location: Bursa/T?RKIYE
Posted: 24th Nov 2021 14:52
I cahnged
+ Code Snippet
If GetFileExists("filename") Then DeleteFile("filename")
CreateFileFromMemblock("filename", mem)


To
+ Code Snippet
//If GetFileExists("filename") Then DeleteFile("filename")
CreateFileFromMemblock("filename", mem)


The server is alive more than a day now.

So, the issue is fixed for me but the bug remains.
Createfilefrommemblock is innocent by the way.
A
adambiser Joined: 16th Sep 2015
Location: US
Posted: 27th Nov 2021 5:45
On Windows, DeleteFile calls "DeleteFileW". The DeleteFile function marks a file for deletion on close. Therefore, the file deletion does not occur until the last handle to the file is closed. Subsequent calls to CreateFile to open the file fail with ERROR_ACCESS_DENIED."
It's possible that CreateFileFromMemblock is happening before the OS has a chance to actually process the delete. It's a race condition. Related

Doesn't explain the blank error message though.
A
Arch-Ok Joined: 11th Jul 2019
Location: Bursa/T?RKIYE
Posted: 29th Nov 2021 15:43
yeah It came to my mind too and I tried

deletfile(file)
do
if getfileexist(file) = 0 then exit
loop
createfilefrommemblock(file)

it crushed again after 7 - 8 hours with blank error message again.

it is about a forgetten file handle to be released i guess...?

luckily 'createfilefrommemblock()' changed the whole file even it exists before, without checking file existence and then deleting it.
A
Arch-Ok Joined: 11th Jul 2019
Location: Bursa/T?RKIYE
Posted: 29th Nov 2021 22:35
Ok, it is official all CreateFileFromMemblock(), GetFileExists() and DeleteFile() are buggy!

+ Code Snippet
i = Creatememblock(1024)
do
	CreateFileFromMemblock("somefile.txt", i)
    Print( ScreenFPS() )
    Sync()
loop


run this code, open task manager, go to details tab and enjoy how this mini code consumes the ram.
Then add GetFileExists() and see how the consuming speeds up!
Then add the DeleteFile() and you will clearly see, they are memory consuming, at some point some kind of handles are not released i guess...

It took so long now, someone in charge must fix these already!
G
Game_Code_here Joined: 2nd Jun 2020
Location:
Posted: 29th Nov 2021 23:27
i = Creatememblock(1024)

Creating a memory block with 1024 mg of memory
CreateMemblock( memID, size )

Creating a file with the size of 1024 mg of memory
CreateFileFromMemblock("somefile.txt", i)


I can see how this would slow down any ones program.

As I read it you are creating a large memory block and then creating a file with that memory.
A
Arch-Ok Joined: 11th Jul 2019
Location: Bursa/T?RKIYE
Posted: 2nd Dec 2021 21:57
Update;
creatememblockfromfile() does the same thing and uses more and more memory after every call.

try this code and open task manager and go to details tab and watch for yourselves;

you need to create a "a.txt" file in the project's media folder.

+ Code Snippet
// Project: creatememblockfromfile 
// Created: 2021-12-03

// show all errors
SetErrorMode(2)

// set window properties
SetWindowTitle( "creatememblockfromfile" )
SetWindowSize( 1024, 768, 0 )
SetWindowAllowResize( 1 ) // allow the user to resize the window

// set display properties
SetVirtualResolution( 1024, 768 ) // doesn't have to match the window
SetOrientationAllowed( 1, 1, 1, 1 ) // allow both portrait and landscape on mobile devices
SetSyncRate( 30, 0 ) // 30fps instead of 60 to save battery
SetScissor( 0,0,0,0 ) // use the maximum available screen space, no black borders
UseNewDefaultFonts( 1 ) // since version 2.0.22 we can use nicer default fonts



do
    i = CreateMemblockFromFile("a.txt")
	Deletememblock(i)
    Print( ScreenFPS() )
    Sync()
loop
S
Scraggle Joined: 10th Jul 2003
Location: Yorkshire
Posted: 2nd Dec 2021 22:04
That one is obviously the expected result.
If you continually create a memblock (set aside an area of memory to write to) then if course it's going to continually increase the amount of memory it uses. That's exactly what you are asking it to do.

Edit
Sorry, I didn't see the DeleteMemblock() call. That changes things