Page 2 of 14

Re: LEV files revisited

Posted: Mon Jan 21, 2013 9:02 am
by PostalDude
Last online on the 7th.....damn.

Re: LEV files revisited

Posted: Mon Jan 21, 2013 12:45 pm
by Fireboyd78
Probably got caught up in things, no big deal. :P

Re: LEV files revisited

Posted: Mon Jan 21, 2013 2:56 pm
by Krishty
Sorry for the silence. Yep; the polygon IDs are quite complicated, and I currently just don't have time to find the required patterns :( For example, I need to see if the ID's are somehow similar to those in the Driver Texture Editor …

I'll probably have more time next week. I'm not logged in, but I'm checking the forum on a daily basis.

In the meantime, here's some hundreds meshes I extracted. You may find buildings, trains, poles, etc. They're from Chicago, Havana, Vegas and some may be of Rio. Have fun.
some meshes.zip

Re: LEV files revisited

Posted: Mon Jan 21, 2013 7:22 pm
by RacingFreak
Awesome, still no luck for cars tho? We can anyway use these buildings to recreate the Driver 2 levels... imagine all of them in Driver (still needs advanced level editor for that :P)

PS
Why you don't try with Driver 2/Driver 1 beta demos? They have less model files to bug with at first. If you need help finding the CD's, let me know

Re: LEV files revisited

Posted: Tue Jan 22, 2013 3:10 am
by Krishty
Betas proved difficult with other games, as developers often leave junk in there, and then we're staring at, it trying to figure out how to parse it, just to see it's useless. I'd rather not do it, and in the current situation there's lots of other things to do.

Re: LEV files revisited

Posted: Tue Jan 22, 2013 3:59 am
by RacingFreak
Alright, sounds ok. Any chances for vehicle models poppin'?

Re: LEV files revisited

Posted: Tue Jan 22, 2013 4:22 am
by Krishty
I really can't say. I have only understood a small fraction of the data yet.

Re: LEV files revisited

Posted: Tue Jan 22, 2013 5:23 am
by RacingFreak
Alright, keep us in touch :)

I did take a look at the models you provided. Any chances for UV mapping ?

Re: LEV files revisited

Posted: Tue Jan 22, 2013 6:30 am
by Krishty
High chance, but probably one model at a time until I decode the IDs.

Re: LEV files revisited

Posted: Tue Jan 22, 2013 6:57 am
by RacingFreak
Okay fair enough :)

Re: LEV files revisited

Posted: Tue Jan 22, 2013 10:44 pm
by Maverick
You're doing a great job on this. Thanks for keeping us posted :)

Re: LEV files revisited

Posted: Fri Aug 02, 2013 9:33 pm
by Krishty
Sorry it's been so long since the last update, but well, here is something small for compensation.

Using someone 972s Driver Texture Editor, I extracted the cities of Driver 1 to vertex lists, and wrote a small tool to inspect the result. You can see a video of last week's version here:
as well as some snapshots here and olders here.

Not much of my own work in it; to great extent I just used DTE's algorithms. I have found many similarities with Driver 2, though; and I may be able to crack it some time in the future.

(Yes transparency didn't work in the video; it does now. So do shadows, and bad geometry has been fixed as well. It's an old version :-( )

Re: LEV files revisited

Posted: Fri Aug 02, 2013 10:38 pm
by Olanov
Awesome!

Re: LEV files revisited

Posted: Sat Aug 03, 2013 12:52 am
by 0takumetalhead
This is really amazing man :D

Re: LEV files revisited

Posted: Sat Aug 03, 2013 4:34 am
by PostalDude
Sweet Zombie Jesus.
That is AWESOME.

Re: LEV files revisited

Posted: Sat Aug 03, 2013 6:30 am
by Skylabh
Great job. Too bad the 2 links are in german language. Not easy to understand it even with a translator lol, and even if i have studied this language over two years, when i was young.

Re: LEV files revisited

Posted: Sat Aug 03, 2013 7:20 am
by Krishty
Olanov wrote:Awesome!
0takumetalhead wrote:This is really amazing man :D
PostalDude wrote:Sweet Zombie Jesus.
That is AWESOME.
Skylabh wrote:Great job.
Thank you!
Skylabh wrote:Too bad the 2 links are in german language. Not easy to understand it even with a translator lol, and even if i have studied this language over two years, when i was young.
Same here with french ;-) I'm just restating some facts we already know from Driver Texture Editor:
Disturbing: what required a high-end PC 14 years ago, I can now render without culling as non-indexed triangle list. At 200 fps. (My system is four years old, so it's rather ten years.)
Driver has also been a PlayStation game, and you can tell this from its data structure. I took time for it because I'm stuck decoding Driver 2 data (which is PlayStation-exclusive).

Most of this has been found by someone972 for his Driver Texture Editor, which I pretty much built upon:

The cities are saved as collections of blocks (because memory is spare and seeking on the CD drive is slow). The New York level seen on the snapshots, for example, is 4×4 km in size, consisting of 8×8 blocks.

Each block references a global model list. The list is 1030 models large for New York, and each model is instantated at any position with an optional rotation about Y, but without scaling.

Models covering more than one block are stored to an additional global block. So in fact it's 65 blocks for New York.

The models have faces, but not necessarily vertices. This is because many buildings differ in color and texture only, but not in geometry. In such cases, models point to vertices of other models. Furthermore, they have bounding boxes and clipping planes. Finally, there is motion capture data for animating people and paths for simulating traffic.

The faces are stored in an awful format: as triangles or quads; with or without texture; with or without vertex color; with or without normals. Of those there is, depending on shading mode, one or two or four. Each combination of these attributes is a unique format and requires dedicated code. I don't know why this has been done on such weak platforms like PlayStation, and whether it was obfuscation. But it was certainly continued for Driver 2.

Because Driver takes place at night often, many faces have normals. In the actual game, a point light is attached to the player.

The vertices are stored as floats in the PC version. But the coordinates are perfectly even and scaled by 256. I therefore assume, the PlayStation version uses 16 or 24 bit fixed-point numbers. Angles and normals are stored in 13 signed bits.

Textures are 8 or 16 bits per pixel. 8-bit textures are palettized, and have a set of up to eight palettes for different times of day and tasks — a tree texture, for example, is recycled as a tree shadow texture with a pitch black palette; cars are identical but their palettes differ for different paintings; and so on. 16-bit textures are RGB or RGBA or none of those; I'm not sure yet.

All textures are 256² (which is typical for PlayStation), and must be considered collections of regions (so-called texture windows). The hardware clamps filtering to these regions at drawing so their neighbours don't bleed in. Emulating this is quite a challenge and will keep me busy; that's why the textures are so ugly on my snap shots.

New York is built from 70,000 model instances with about 300,000 triangles in total. Currently, I'm combining all faces to a 40 megabyte vertex list, where I also store each face's texture index. The 170 textures are converted to 24-bit RGBA and stored in a 256×256×170 texture array (about 30 MiB total). The city is drawn in a single draw call; the texture index from the face is passed from the vertex data through the vertex shader to the pixel shader to be looked up in the texture array.

The three other cities (Miami, Los Angeles, San Francisco) have similar dimensions and data sizes. The apartment from the cut scenes is much smaller. I could not load the training level yet; the format does not play out exactly.
I'm sure Driver Texture Editor helped you a lot (I had a look at it (at its source)).
Indeed; especially the face decoding has saved days of work for me. It's a pity I didn't find this tool earlier.
It's nice of you always sharing your knowledge in this forum. Thanks again.
You're welcome; you see, for obvious copyright problems I can't just put it up for download.

In the meantime:

I mostly solved the transparency issues: paletted textures are always opaque; in 15-bit RGB textures, the color key (1, 1, 1) [0x4321 hex] denotes full transparency. I have yet to understand what the 16th color bit is for. It may be an additional bit of accuracy for one of the color channels.

Furthermore, I perform some trivial lighting calculation on the surfaces. The effect is massive though, because linear color space shading makes the haze very appearent in front of shady places.

Re: LEV files revisited

Posted: Sat Aug 03, 2013 10:02 am
by Skylabh
Thanks for the translation.

Re: LEV files revisited

Posted: Sat Aug 03, 2013 5:37 pm
by max.thunder
Very interesting read, great job!

Re: LEV files revisited

Posted: Sat Aug 03, 2013 6:23 pm
by Krishty
Some snapshots of San Francisco:

Image
Image
Image
Image
Image

Sorry for the low contrast, but the haze is still configured for New York.

Looking at the dimensions and the quality of cities, I guess Training and L.A. were produced first. They're the smallest, and their visual quality is lowest. Miami was already better, but blocky in many places. NY and Frisco are of similar quality, but New York is more optimized (less stray polygons).

Re: LEV files revisited

Posted: Sat Aug 03, 2013 6:38 pm
by Submaniac
Woow! This is so nice! Imagine if we could import those cities as mods in other games! :O It would be so. AWESOME :D

Re: LEV files revisited

Posted: Sat Aug 03, 2013 7:04 pm
by 0takumetalhead
Wow, more stuff to show us and an great read on how the game handles it cities, a really interesting double whammy.
16 square kilometer's per city is pretty big man, at least for a PS1 game :O
Pretty mindblowing to know how such a big sized city can be handled by 4 to 6 rams of memory, nevermind Director Mode to boot!

Re: LEV files revisited

Posted: Sat Aug 03, 2013 7:16 pm
by Krishty
0takumetalhead wrote:16 square kilometer's per city is pretty big man, at least for a PS1 game :O
Pretty mindblowing to know how such a big sized city can be handled by 4 to 6 rams of memory, nevermind Director Mode to boot!
Yes, but you need to consider not all 16 km² are covered. In NY, only 10 are actually used, and the others are empty. I guess Driver 2's cities are even larger, and I am very keen on extracting them :-)

Also, much of this has been known for long from DTE2's source code.

I have some more screenshots in the pipeline, stay tuned.

Re: LEV files revisited

Posted: Sat Aug 03, 2013 7:28 pm
by 0takumetalhead
Krishty wrote:Yes, but you need to consider not all 16 km² are covered. In NY, only 10 are actually used, and the others are empty. I guess Driver 2's cities are even larger, and I am very keen on extracting them :-)

Also, much of this has been known for long from DTE2's source code.

I have some more screenshots in the pipeline, stay tuned.
Well I figured out sofar LA isn't using all of the availble size (most likely only half of it). Anyway still an impressive feat even when not all of the availble size is used, since there are enough other PS1 games that can't make such a great use of the limited RAM memory.

Re: LEV files revisited

Posted: Sat Aug 03, 2013 7:42 pm
by Krishty
Yes, but from what I see, there is still much headroom for improvements.

I'm keen on Driver 2's files: On the one hand, Driver 2 outperformed Driver 1 in size and details; but on the other hand I remember an interview with a lead programmer who stated they were dissatified with its performance and had to finish it off quickly for christmas selling.

Anyway, here are garages:
Image
Image
Image

Re: LEV files revisited

Posted: Sat Aug 03, 2013 7:45 pm
by Krishty
The flats:
Image
Image
Image
Image

Re: LEV files revisited

Posted: Sat Aug 03, 2013 7:46 pm
by Krishty
… and I can now proudly announce I got the training level loading:
Image

Re: LEV files revisited

Posted: Sat Aug 03, 2013 7:48 pm
by bb_42001
Love the newspapers name on the first picture " Duh", Then the headline " This is the most dangerous man"

Re: LEV files revisited

Posted: Sat Aug 03, 2013 7:52 pm
by Krishty
bb_42001 wrote:Love the newspapers name on the first picture " Duh", Then the headline " This is the most dangerous man"
Yes! There is also some irony in New York:

Image

Re: LEV files revisited

Posted: Sat Aug 03, 2013 9:48 pm
by Skylabh
Really awesome.