Page 6 of 14

Re: LEV files revisited

Posted: Sat Jul 05, 2014 6:04 pm
by someone972
Ya, that's probably the "modelDefs" offset. If you seek 2048*contentsTableSize+2048*contentsNodesSize instead of 2048*5, I think it should probably get you to that spot reliably. Note for Driver 2 you'll have to add an offset to each sector if you are drawing them, because it stores the model def positions relative to the sector origin so that it can use shorts instead of longs for position. Let me know if using the offset I said above gets you to the right spot or not.

Re: LEV files revisited

Posted: Sat Jul 05, 2014 6:25 pm
by kierowca12
It doesn't work. If I add 2048*5 7 piece map are good, if I add 2048*contentsNodesSize 5 are good(multiplayer chicago map).

Re: LEV files revisited

Posted: Sat Jul 05, 2014 6:27 pm
by someone972
Wait, so did you add just 2048*contentsNodesSize or 2048*contentsNodesSize+2048*contentsTableSize?

Re: LEV files revisited

Posted: Sat Jul 05, 2014 10:33 pm
by kierowca12
My mistake, 2048*contentsNodesSize+2048*contentsTableSize work.

Re: LEV files revisited

Posted: Sun Jul 06, 2014 6:08 pm
by someone972
I was able to load the overlay map, after much annoyance. Below is the map for multiplayer Chicago. Apparently it still has the whole overlay map even though it only shows a portion of it. I'll update the spec with the format when I have time.

Image

Re: LEV files revisited

Posted: Mon Jul 07, 2014 3:01 am
by VAIMAHDO
Yay! The map! And why is it only the multiplayer game files you're able to extract content?

Re: LEV files revisited

Posted: Mon Jul 07, 2014 6:10 am
by someone972
It's not that we can't extract it from other levels, it's just that the multiplayer levels are significantly smaller, so decoding the format for most things is a lot easier.

Re: LEV files revisited

Posted: Wed Jul 09, 2014 4:44 pm
by someone972
I've updated the spec to include the Car Models block, and I also made a fix to the Sector Info block (the shared groups structure was wrong). Also, I forgot to mention that if you seek to modelsOffset*2048+(modelsSize-1)*2048, there is a list of model indices starting with a dword count. You'll need this to properly load the models from the shared group. Eventually I'll add the information about it to the spec, but I've been a bit thin on time lately.

Re: LEV files revisited

Posted: Thu Jul 10, 2014 6:19 am
by SOAP
someone972 wrote:I've updated the spec to include the Car Models block, and I also made a fix to the Sector Info block (the shared groups structure was wrong). Also, I forgot to mention that if you seek to modelsOffset*2048+(modelsSize-1)*2048, there is a list of model indices starting with a dword count. You'll need this to properly load the models from the shared group. Eventually I'll add the information about it to the spec, but I've been a bit thin on time lately.
I have successfully loaded car models, and exported them =)
http://www.mediafire.com/download/c65h6 ... models.rar

I've realized out why some faces have invalid indices: some flags (or face types) didn't include texture index and palette index (faces with 0 flags are 8 bytes, in car models)
Later i'll try to figure out how other bitflags depends on face size and commit changes to git

EDIT:
car models in archive have wrong texture coordinates

Re: LEV files revisited

Posted: Thu Jul 10, 2014 11:28 am
by Skylabh
Nice work (Y)

Re: LEV files revisited

Posted: Fri Jul 11, 2014 1:32 am
by SOAP
I did it
Image

And also here cars from Driver 1
Textures aren't included
http://www.mediafire.com/download/416ux ... MODELS.rar

Seems to be faces are mostly parsed correctly
here is compiled tool
http://www.mediafire.com/download/gcig7 ... l_tool.rar

also made commit to my github
https://github.com/SoapyMan/psx_driver2_lev

Re: LEV files revisited

Posted: Fri Jul 11, 2014 3:31 am
by RacingFreak
YOU DID IT!

Thanks SOAP once again! :specialdriver:

Re: LEV files revisited

Posted: Fri Jul 11, 2014 3:36 am
by Klancnik777
dasitmane :D

noice work

Re: LEV files revisited

Posted: Fri Jul 11, 2014 4:07 am
by SOAP
RacingFreak wrote:YOU DID IT!

Thanks SOAP once again! :specialdriver:
Big thanks to someone972 for specs!

Re: LEV files revisited

Posted: Fri Jul 11, 2014 9:51 am
by VAIMAHDO
Lol I spent hours last night re setting up textures and re texture mapping cars. Also cleaning up how the textures are mapped on because some of the textures seemed to have been placed on a bit sloppy.
For example, does anyone remember how this car's driver and passenger door windows were a bit bent? Well I fixed that!

Post edited by Skylabh.
Reason: picture no longer available.


New:

Post edited by Skylabh.
Reason: picture no longer available.


This car is using my own NON mirrored texture layout, your all welcome to use it:

Post edited by Skylabh.
Reason: picture no longer available.


Also, I never noticed Driver 2's cars have mirrors! (Probably mostly due to the fact that I've always played the game on the actual PS1 console in the original low resolution pixel rendering. Also guys I can thank you for extracting these cars, they will help me master and how do deal with weak machines (And if I decide to make a PS1 game) and discover how polygons were distributed at the time.

Re: LEV files revisited

Posted: Fri Jul 11, 2014 2:22 pm
by RacingFreak
That's pretty cool, but don't forget UV mapping is split on segments for side texture :wink:
SOAP wrote:
RacingFreak wrote:YOU DID IT!

Thanks SOAP once again! :specialdriver:
Big thanks to someone972 for specs!
Yes of course, everyone worked hard on this to happen, but you were able to build it altogether and shared these goodies with us :specialdriver:

I can't wait to see more progress! (Y)

I might get into making some models the coming days :D

Re: LEV files revisited

Posted: Sat Jul 12, 2014 11:43 pm
by kierowca12
Can you encode models from SharedDataGroup.modelsOffset?
Anybody could publish code to encode textures?

Re: LEV files revisited

Posted: Sun Jul 13, 2014 5:55 am
by Krishty
I don't get Sector Info (Block Identifier: 26 (0x1a)). For multiplayer Chicago:
Code: Select all
1A 00 00 00 < ID
3C 06 00 00 < size

24 00 textureOffset
00 00 modelsOffset … uh?!
20 00 unk1
00 00 unk2
00    modelsSize
00    unk3
00    numTextures … wat
00    unk4
00 00 unk5
00 00 unk6
How is this supposed to work?

Edit: Nevermind – I should have read it from the bottom up. It's the "Sector Info Format" that applies. Stupid me! :)

Re: LEV files revisited

Posted: Sun Jul 13, 2014 6:59 am
by Krishty
SOAP wrote:LUMP_SPOOLINFO is something mystical, but tried to decompile function that loads offsets
It's difficult to understand MIPS assembly sometimes :D

Procedure is directly taken from driver 2 demo executable, and decompiled to C
It loads spool info (may be partial, because cell allocation function call is next after this)
Code: Select all
struct struct_0 {
    int32_t e0[4];
    int32_t e1;
};

int32_t * g1; // 0xa41e8
struct struct_0 g2; // 0xa41f8
struct struct_0 g3; // 0xa4620
int32_t g4 = 0; // gpregs4 - initially lump data pointer

void function_40000(void) {
    int32_t v1 = g4; // 0x40000
    int32_t v2 = *(int32_t *)0x94ec0; // 0x40010
    g4 = v2;
    int32_t v3 = 2048 * *(int32_t *)v1; // 0x40014
    *(int32_t *)3076 = v2;
    int32_t v4 = v1 + 4;
    int32_t v5; // 0x4002c
    if ((int32_t)(v3 > 0xffff) == 0) {
        // 0x40024
        v5 = 0x10000;
        // branch -> 0x4002c
    } else {
        // .dec_label_pc_4002c_crit_edge
        v5 = v3;
        // branch -> 0x4002c
    }
    int32_t v6 = g4 + v5; // 0x4002c
    int32_t v7 = v6;
    int32_t v8 = 0xa41e8;
    *(int32_t *)0x94ec0 = v6;
    *(int32_t *)3068 = v4 + 4;
    int32_t v9 = *(int32_t *)v4 + 4 + v4; // 0x40074
    g4 = *(int32_t *)v9;
    *(int32_t *)3020 = v9 + 4;
    int32_t v10 = 16 * g4; // 0x40084
    int32_t v11 = (v10 | 4) + v9; // 0x4008c
    *(int32_t *)3088 = v11;
    int32_t v12 = v10 + v11; // 0x40094
    *(int32_t *)3040 = g4;
    g1[v8] = v7;
    int32_t v13 = 0; // 0x400a4
    int32_t v14 = 0xa4620;
    g3.e0[v14] = v13;
    int32_t v15 = 0; // 0x400a8
    int32_t v16 = 0xa41f8;
    g2.e0[v16] = v15;
    int32_t v17 = v12; // 0x400ac
    int32_t v18 = v8 + 4; // 0x400c4
    int32_t v19 = 2; // 0x400c8
    int32_t v20 = *(int32_t *)v17 + v13; // 0x400cc
    int32_t v21 = *(int32_t *)(v17 + 16) + v15; // 0x400d0
    int32_t v22 = v7 + (*(int32_t *)(v17 + 32) + 2047 & -2048); // 0x400dc
    // branch -> 0x400a0
    while (v19 > 0xffffffff) {
        // 0x400a0
        g1[v18] = v22;
        v13 = v20;
        v14 += 4;
        g3.e0[v14] = v13;
        v15 = v21;
        v16 += 4;
        g2.e0[v16] = v15;
        v17 += 4;
        v18 += 4;
        v19--;
        v20 = *(int32_t *)v17 + v13;
        v21 = *(int32_t *)(v17 + 16) + v15;
        v22 += (*(int32_t *)(v17 + 32) + 2047 & -2048);
        // continue -> 0x400a0
    }
    // 0x400e0
    v4 = v12 + 48;
    g3.e1 = v20;
    g2.e1 = v21;
    *(int32_t *)0x94ec0 = v22;
    *(int32_t *)3048 = v4 + 4;
    *(int32_t *)2968 = 2 * *(int32_t *)v4 + 4 + v4 + 4;
}
Looks like i've found a nice MIPS translator at http://decompiler.fit.vutbr.cz (code structure only, data types and numbers may be wrong)
Other decompilers which i've tried aren't capable to generate valid code (even asm in IDA didn't look good while i've tried to translate it manually)

I think this code might be helpful. I'll test it soon to load lump properly
This is my interpretation:
Code: Select all
int32_t g1[5]; // 0xa41e8
int32_t listB[5]; // was "g2"
int32_t listA[5]; // was "g3"

int32_t g4 = 0; // gpregs4 - initially lump data pointer

void function_40000(void) {
    int32_t dataptr = g4; // 0x40000
    int32_t v2 = *(int32_t *)0x94ec0; // 0x40010
    g4 = v2;
    int32_t texOrModelOffset = 2048 * *(int32_t *)dataptr; // 0x40014  ergibt 0x12000
    *(int32_t *)3076 = v2;
    dataptr += 4;

	if(texOrModelOffset < 0x10000) {
		texOrModelOffset = 0x10000;
	}

    int32_t v7 = g4 + texOrModelOffset;
    *(int32_t *)0x94ec0 = v7;
    *(int32_t *)3068 = dataptr + 4;
    int32_t v9 = *(int32_t *)(2 * dataptr + 4); // THIS DOES NOT MAKE SENSE
    g4 = *(int32_t *)v9;
    *(int32_t *)3020 = v9 + 4;
    int32_t v10 = 16 * g4; // 0x40084
    int32_t v11 = (v10 | 4) + v9; // 0x4008c
    *(int32_t *)3088 = v11;
    int32_t v12 = v10 + v11; // 0x40094
    *(int32_t *)3040 = g4;

    int32_t offsetOfA = 0;
	int32_t offsetOfB = 0;
    struct V17Data {
		int32_t a[4];
		int32_t b[4];
		int32_t c[4];
	} v17 = v12;
	for(int i = 0; i < 4; ++i) {
		g1[i] = v7;
        listA[i] = offsetOfA;
        listB[i] = offsetOfB;

		v7 += roundUpTo2048(v17.c[i]);
        offsetOfA += v17.a[i];
        offsetOfB += v17.b[i];
	}
    g1[4] = v7;
    listA[4] = offsetOfA;
    listB[4] = offsetOfB;
    dataptr = v12 + sizeof(V17Data); // means + 48
	
    *(int32_t *)3048 = dataptr + 4;
    *(int32_t *)2968 = 2 * *(int32_t *)dataptr + 4 + dataptr + 4; // MAKES NO SENSE TO ME
}
I.e. there's 3×4 ints somewhere in the file, and they're loaded to compute addresses of 3×5 things in a global list. The first list of things is 2048-B-aligned. Could be that's the 12 unknown ints in the spool lump. I hope it helps.

Re: LEV files revisited

Posted: Sun Jul 13, 2014 7:04 pm
by Krishty
someone972 wrote:Also, I forgot to mention that if you seek to modelsOffset*2048+(modelsSize-1)*2048, there is a list of model indices starting with a dword count. You'll need this to properly load the models from the shared group. Eventually I'll add the information about it to the spec, but I've been a bit thin on time lately.
Okay, so my view on the list:
1) the list begins with a word which gives the number of words following immediately
2) this is also the number of models in the sector
3) the global model list at the beginning of the file has lots and lots of entries without geometry
4) each index in the word list points to a geometry-less model in the global model list
5) the according models in the sector fit into the global model list exactly

It's getting better and better:
Image
But why are the streets rotated upright? :-(

Edit: The vertex references of models must be resolved *after* a section's models have been inserted into the global model list. I'm working on it.

Re: LEV files revisited

Posted: Sun Jul 13, 2014 8:25 pm
by kierowca12
Krishty wrote:But why are the streets rotated upright? :-(
It looks awesome. We are so close. :)
Could you share with us your code?

Re: LEV files revisited

Posted: Sun Jul 13, 2014 9:15 pm
by Krishty
I'll share when textures are working :) In the meantime, feel free to ask me.

Here's a bit of Havana:

Image

Anybody knows where to get the number of XYZ/rot+index structures from? I'm just guessing 2000 per tile for the moment …

Re: LEV files revisited

Posted: Sun Jul 13, 2014 11:35 pm
by RacingFreak
Amazing work!! Keep it up (Y)

Re: LEV files revisited

Posted: Sun Jul 13, 2014 11:40 pm
by Klancnik777
this is beutifull ;)

Re: LEV files revisited

Posted: Mon Jul 14, 2014 12:26 am
by Krishty
I found the texture data, and it looks almost correct, except that texture coordinates are wrong. The same pixel is used all over the polygons (as if each vertex had the same texture coordinate). This is really killing me now.

Re: LEV files revisited

Posted: Mon Jul 14, 2014 12:58 am
by Krishty
Any idea where the texture coordinates are?

Image

Roads etc. are okay. Texture indices are alright, too (the building colors would be wrong otherwise). But where the f*** are those texture coordinates?!

Re: LEV files revisited

Posted: Mon Jul 14, 2014 1:19 am
by SOAP
Krishty wrote:Any idea where the texture coordinates are?

{IMAGE CUT}

Roads etc. are okay. Texture indices are alright, too (the building colors would be wrong otherwise). But where the f*** are those texture coordinates?!
I'll check them. Face decoding is headache

EDIT:
realized that some faces (basically quads) uses one bitflag that defines palette and texture, but texture coordinates are taken from TextureInfo

Re: LEV files revisited

Posted: Mon Jul 14, 2014 3:01 am
by Krishty
What I tried so far:
Code: Select all
if((0 == out->flags & FACE_IS_TEXTURED) && (face->flags & FACE_IS_QUAD)) {
  quad.rgn = atlases[out->texture].region[out->palette];
}
The atlas is correct (verified for a few cases), but the region is f***ed up entirely:
Image

Re: LEV files revisited

Posted: Mon Jul 14, 2014 3:07 am
by SOAP
Krishty wrote:What I tried so far:
Code: Select all
if((0 == out->flags & FACE_IS_TEXTURED) && (face->flags & FACE_IS_QUAD)) {
  quad.rgn = atlases[out->texture].region[out->palette];
}
The atlas is correct (verified for a few cases), but the region is f***ed up entirely:
Image
I think i've found right bit!!! And there is no faces that have both FACE_TEXTURED_COORDS and FACE_TEXTURED_ATLAS flags
Code: Select all
enum EFaceFlags_e
{
	FACE_IS_QUAD			= (1 << 0),
	FACE_RGB				= (1 << 1),	// this face has a color data
	FACE_TEXTURED_COORDS	= (1 << 2),	// this face is textured and has custom texture coordinates
	FACE_TEXTURED_ATLAS		= (1 << 3),	// this face is textured, and texcoords are taken from texture atlas

	FACE_HAS_VERTEXNORMAL	= (1 << 4),     // still questionable

	FACE_UNK6				= (1 << 5),
	FACE_UNK7				= (1 << 7),
	FACE_UNK8				= (1 << 8),
};

inline int decode_face(const char* face, dface_t* out)
{
	char* data = (char*)face;

	out->flags = *data++;

	int numComp = 4;

	if((out->flags & FACE_TEXTURED_COORDS) || (out->flags & FACE_TEXTURED_ATLAS) || (out->flags & FACE_IS_QUAD))
	{
		out->palette = *data++;
		out->texture = *data++;
		*data++;
	}
	else
		numComp = (out->flags & FACE_IS_QUAD) ? 4 : 3;

	for(int i = 0; i < numComp; i++)
	{
		out->vindex[i] = *data++;
	}

	// read color - ALWAYS 4 bytes
	if(out->flags & FACE_RGB)
	{
		for(int i = 0; i < 4; i++)
		{
			out->color[i] = *data++;
		}
	}

	if(out->flags & FACE_TEXTURED_COORDS)
	{
		for(int i = 0; i < 4; i++)
		{
			out->texcoord[i][0] = *data++;
			out->texcoord[i][1] = *data++;
		}
	}
	
	// PADDING?
	if(out->flags & FACE_HAS_VERTEXNORMAL && numComp == 3 && !(out->flags & FACE_RGB))
	{
		// read normals
		for(int i = 0; i < 4; i++)
		{
			out->nindex[i] = *data++;
		}
	}
	
	// read something (???) - ALWAYS 4 bytes
	for(int i = 0; i < 4; i++)
	{
		out->nindex[i] = *data++;
	}

	return data-(char*)face;
}

Re: LEV files revisited

Posted: Mon Jul 14, 2014 3:36 am
by Krishty
Awesome! Thank you very much (Y)

Image

Now I'll need to figure out the correct texture coordinates so the quads don't rotate …