AppGameKit Classic Chat->[SOLVED] Parsing JSON files longer than the string size limit
D
Dualunitfold Joined: 13th Oct 2019
Location:
Posted: 1st Dec 2023 18:30
I'm trying to read maps from Tiled into my game, and I've hit an interesting roadblock: the json is now longer than the string data type allows.
The only way I can find of parsing the json is by passing the whole thing as a string into the .fromJSON function of a user-defined type. Is there a way I could get around the string size limit other than splitting out the actual map data into a separate csv file and having the json point to that?

Here's the code I'm currently using to read the map data from the json file:

+ Code Snippet
TYPE TiledRootType
	compressionlevel as integer
	height as integer
	infinite as integer
	layers as TiledLayerType []
	nextlayerid as integer
	nextobjectid as integer
	orientation as string
	renderorder as string
	tiledversion as string
	tileheight as integer
	tilesets as TiledTilesetType []
	tilewidth as integer
	_type as string
	version as string
	width as integer
ENDTYPE

TYPE TiledLayerType
	data as integer []
	height as integer
	id as integer
	name as string
	opacity as integer
	_type as string
	visible as integer
	width as integer
	x as integer
	y as integer
ENDTYPE

TYPE TiledTilesetType
	firstgid as integer
	source as string
ENDTYPE

FUNCTION ReadMap(MapToPopulate ref as MapType, Filename as string)
	JsonData as string
	file = OpenToRead(Filename)
	WHILE NOT FileEOF(file)
		JsonData = JsonData + ReadLine(file)
	ENDWHILE
	Root as TiledRootType
	Root.fromJSON(JsonData)
	MapToPopulate.tiles.length = Root.layers[0].width - 1
	FOR i = 0 TO MapToPopulate.tiles.length
		MapToPopulate.tiles[i].length = Root.layers[0].height - 1
	NEXT i
	FOR i = 0 TO Root.layers[0].data.length
		MapToPopulate.tiles[Mod(i, Root.layers[0].width), Floor(i/Root.layers[0].width)].collision = Root.layers[0].data[i]
	NEXT i
	CloseFile(file)
ENDFUNCTION
B
blink0k Joined: 22nd Feb 2013
Location: the land of oz
Posted: 2nd Dec 2023 0:02
Why not just use JsonStructure.load("filename")
H
hendron Joined: 20th Dec 2015
Location:
Posted: 2nd Dec 2023 16:51
What error message are you getting, if any? Can you provide the Json file in question? Out of curiosity, what happens if you replace

+ Code Snippet
    WHILE NOT FileEOF(file)
        JsonData = JsonData + ReadLine(file)
    ENDWHILE


with

+ Code Snippet
JsonData = ReadString(file)


You shouldn't have to read through a Json file line by line to load it into a string. At least, I've never needed to (including with Tiled Json files). I wonder if that could be causing a problem.

Why not just use JsonStructure.load("filename")


Can you elaborate? This doesn't appear to exist in Tier 1.
V
Virtual Nomad Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 2nd Dec 2023 18:00
Can you elaborate? This doesn't appear to exist in Tier 1.

see JSON near the bottom of the V2 Arrays guide.

then, i'm curious how AppGameKit handles that functionality vs the string limitation.

meanwhile, i don't believe the .json is binary so ReadString() ?
H
hendron Joined: 20th Dec 2015
Location:
Posted: 2nd Dec 2023 18:26
Ah, I see. It's for arrays and doesn't work on UDTs directly. I guess a workaround would be to wrap the UDT and Json data in a single element array.

meanwhile, i don't believe the .json is binary so ReadLine() would be appropriate vs ReadString() ?


I've always used ReadString(file) to read in an entire Json file. Used it for my own Tiled and Ogmo importer modules in the past. Definitely works.

+ Code Snippet
type myJson
    text as string
    int as integer
    real as float
endtype

UseNewDefaultFonts(1)
SetVirtualResolution(1024,768)

file = OpenToRead("tester.json")
Json$ = ReadString(file)
CloseFile(file)

test as myJson
test.fromJSON(Json$)


do
    print(test.text)
    print(test.int)
    print(test.real)
    sync()
loop
V
Virtual Nomad Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 2nd Dec 2023 18:44
I've always used ReadString(file)

ah, so not using While/EndWhile at all (thought you were suggesting replacing just the 1 ReadLine() line). but, that's very cool.

no hands on experience with .json here beyond the documentation so good to know
H
hendron Joined: 20th Dec 2015
Location:
Posted: 2nd Dec 2023 19:12
ah, so not using While/EndWhile at all (thought you were suggesting replacing just the 1 ReadLine() line). but, that's very cool.


Oh yeah, I should have clarified that. Just one call to ReadString() should work for the whole file. Not sure if this will solve OP's problem, though. I do remember running into an issue with FileEOF() not behaving as expected, but this was years ago.
B
blink0k Joined: 22nd Feb 2013
Location: the land of oz
Posted: 2nd Dec 2023 21:37
+ Code Snippet
RootArr as TiledRootType[]
RootArr.load(Filename)
Root as TileRoot
Root = RootArr[0]
D
Dualunitfold Joined: 13th Oct 2019
Location:
Posted: 7th Dec 2023 2:27
Thank you all for the replies! This was for a game jam, so I just used a workaround for that, but I plan on continuing work on the game, so a better solution would be good. I shall try ReadString() and arr.load() and see what results they have.

As for the questions you had, the error it gives is "Error: Invalid JSON, unexpected character in object in MapLoading.agc at line 51

I have also attached the json file which causes this error, called "Map2.json"
The test I did which lead me to the conclusion that I was reaching the string limit was I printed out the result of Right(JsonData) after reading the file, and it was definitely not the end of the file that printed out. It's entirely possible that it could be a limit on some function that I used along the way though rather than a limit on the length of the string (for example FileEOF() acting weirdly)

Cheers!
D
Dualunitfold Joined: 13th Oct 2019
Location:
Posted: 7th Dec 2023 5:09
Alright, I've given both methods a quick test, and the ReadString() method definitely works, so I think I will go forward with that one.
The array.load() method with the json root being a single element array doesn't seem to populate the type correctly. It doesn't error when reading, but if I do Root[0].layers[0] it complains at me saying layers doesn't have any elements, and none of the other attributes have any data in them either

Thanks very much!
H
hendron Joined: 20th Dec 2015
Location:
Posted: 7th Dec 2023 15:06
Glad it worked! My only guess as to why, is that ReadLine() may be messing with the format in some way that the Json parser doesn't like. I don't really know that much about the inner workings, though. ReadString() works by reading characters until it reaches a null terminator (ie 0x00), so it will read a plain text file in its entirety, leaving the formatting untouched. ReadLine() reads until it reaches a line feed character (0x0A) and (I think) removes the character from the string it returns, so the formatting is changed.