Trespasser - CRC-32 hash tools

List your programs here (game-related or other software).

Moderators: scallenger, Rebel, madppiper, TresCom Support Team

User avatar
machf
T-Rex Killer
T-Rex Killer
Posts: 12368
Joined: Thu Apr 24, 2003 11:20 pm
Location: Lima, Peru
Contact:

Re: Trespasser - CRC-32 hash tools

Post by machf »

I just found one myself:

Code: Select all

16019AE1	Bullet - Metal - Res	3A991CB1	(bullet+metal resonate 1)
Which reminds me... it would be worth trying your method with this one, which has never fully convinced me (even though it was one I found myself):

Code: Select all

315	58646391	Terrain - Metal Resn	519226EB
LtSten wrote:I had another run over the "spec-metal" varieties. Possible matches for 97 (66AA7AC2) and 98 (FFA32B78) are "spec-metal insertion01" and "spec-metal insertion02". I've had a listen to them, and they do sound like a metallic thunk, plausibly like something metallic being inserted.
I was trying something related to the Mainframe during my last attempts, as the first part of the sound "SPEC-MAINFRAME BOOT START" (the preceding one) uses that one... but I guess "Insertion" explains a lot. Previously I had tried "Metal lock", "Metal slider" and such ones...
Since those IDs seem to have names in alphabetical order, "insertion01" comes before "switch01", which is promising - although "S" is admittedly rather late in the alphabet anyway.
Oh, it happens. They don't need to be totally contiguous.
Trying to crack 105, 106, and 107 by ear...
I've tried to figure out what sort of machine that could be. I've tried "centrifuge", "manlifter", not sure if "floppy drive"... I think I've run out of ideas for it.

Going back to your earlier post... I never thought "Lindstradt rifle" would be among the other files in that block, which are all automatic rifles. At least now we can narrow down the following two to something between "Lindstradt" and "Ruger"...
Oh, and some other gun also had its order reversed (02 before 01), I think it may have been either the M-16 or one of the S&W revolvers...

When are you going to look at the Ambient and Stream TPA files? Those have fewer unknown names (the ones from earlier builds, I mean - the retail has no unknowns for those).

And this is an interesting sound, the one with the bees, flies or whatever they are:

Code: Select all

453	25B10F6C	[unknown]	6BF9A9FF	 (bees, flies, mosquitoes?)
454	2027364A	[unknown]	81042423	C84A0E90 +”2 METAL”/9BF9D3A0 +“ B METAL” (bullet+metal pinging 1/cattle prod/rifle)
455	7AEF671A	[unknown]	190EDDA7	C84A0E90 +”4 WOOD”/9BF9D3A0 +“ D WOOD” (bullet+soft terrain)
456	86503DFE	[unknown]	B66EE622	C84A0E90 +”3 METAL”/9BF9D3A0 +“ C METAL” (bullet+metal hard 1/paint can/metal soft/terrain - metal/horseshoe/metal crate medium/pistol,metal crate small,metal dull 2)
In the retail, it's located inside the Effects.tpa file, in the very earlier builds (22, 32 and 55) it doesn't exist, and in builds 96, 97, 99 and 103 it's been added to Ambient.tpa after the sounds from the earlier builds, aside from having been added to Effects.tpa, too.

Since in AMbient.tpa it's situated here:

Code: Select all

 205	C8375240	 AMB-BE-B05	44DA51ED
 206	25B10F6C	 [unknown]	6BF9A9FF	 (bees, flies, mosquitoes?)
 207	D5456DB5	AMB-JUNGLE-01	195B46D3
my guess is that it starts with "AMB-" and is between those other two, alphabetically. It may have been added to Ambient.tpa in some build between 55 and 96, then it was also added to Effetcs.tpa, and finally by the time of the retail build, it was removed from Ambient.tpa and only left in Effects.tpa
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files
User avatar
LtSten
Gallimimus
Gallimimus
Posts: 609
Joined: Fri Nov 27, 2009 8:28 pm
Antispam: No
Location: England, UK

Re: Trespasser - CRC-32 hash tools

Post by LtSten »

machf wrote:I just found one myself:

Code: Select all

16019AE1	Bullet - Metal - Res	3A991CB1	(bullet+metal resonate 1)
Which reminds me... it would be worth trying your method with this one, which has never fully convinced me (even though it was one I found myself):

Code: Select all

315	58646391	Terrain - Metal Resn	519226EB
Ah, "Res", yes, never thought of that. Good catch. I'll try to remember to have a look at that one you mentioned.
machf wrote: When are you going to look at the Ambient and Stream TPA files? Those have fewer unknown names (the ones from earlier builds, I mean - the retail has no unknowns for those).
Hopefully at some point over the next week. It sounds like it might be useful to use all of them to generate a word list more suited for Trespasser (currently I only grabbed the known names from the retail TPAs).
I'd also like to get the source uploaded - although at the moment I've been tweaking code to try different tests rather than having any command-line arguments (I never did write a good parser for those...), so it's not all that user friendly. Building from source will also require at least moderate C++20 support (I'm on Visual Studio 16.8.3 for now) since some of it makes use of concepts and ranges. It could be rewritten to be C++17 compatible, but I'll leave that as an exercise to the interested reader. ;)
machf wrote: And this is an interesting sound, the one with the bees, flies or whatever they are:

Code: Select all

453	25B10F6C	[unknown]	6BF9A9FF	 (bees, flies, mosquitoes?)
454	2027364A	[unknown]	81042423	C84A0E90 +”2 METAL”/9BF9D3A0 +“ B METAL” (bullet+metal pinging 1/cattle prod/rifle)
455	7AEF671A	[unknown]	190EDDA7	C84A0E90 +”4 WOOD”/9BF9D3A0 +“ D WOOD” (bullet+soft terrain)
456	86503DFE	[unknown]	B66EE622	C84A0E90 +”3 METAL”/9BF9D3A0 +“ C METAL” (bullet+metal hard 1/paint can/metal soft/terrain - metal/horseshoe/metal crate medium/pistol,metal crate small,metal dull 2)
In the retail, it's located inside the Effects.tpa file, in the very earlier builds (22, 32 and 55) it doesn't exist, and in builds 96, 97, 99 and 103 it's been added to Ambient.tpa after the sounds from the earlier builds, aside from having been added to Effects.tpa, too.

Since in AMbient.tpa it's situated here:

Code: Select all

 205	C8375240	 AMB-BE-B05	44DA51ED
 206	25B10F6C	 [unknown]	6BF9A9FF	 (bees, flies, mosquitoes?)
 207	D5456DB5	AMB-JUNGLE-01	195B46D3
my guess is that it starts with "AMB-" and is between those other two, alphabetically. It may have been added to Ambient.tpa in some build between 55 and 96, then it was also added to Effetcs.tpa, and finally by the time of the retail build, it was removed from Ambient.tpa and only left in Effects.tpa
That's useful to know, I'd mostly disregarded "AMB-" as a prefix. I should have a look at what else I'm perhaps missing when I go through the non-retail lists.
Bringing TPA CRCs up the well, amongst other file format adventures.
Blender 3.0 TPM Plugin [Project GitHub]
User avatar
machf
T-Rex Killer
T-Rex Killer
Posts: 12368
Joined: Thu Apr 24, 2003 11:20 pm
Location: Lima, Peru
Contact:

Re: Trespasser - CRC-32 hash tools

Post by machf »

You know, it's funny how I could have easily found "insertion" just using "spec-metal " as leading string, "01" as trailing string, and only 5 random characters... I may even have tried that combination but missed it among the results in the listing somehow. It only takes a few seconds to generate...
Oh, well, we already found it now anyway.
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files
User avatar
machf
T-Rex Killer
T-Rex Killer
Posts: 12368
Joined: Thu Apr 24, 2003 11:20 pm
Location: Lima, Peru
Contact:

Re: Trespasser - CRC-32 hash tools

Post by machf »

machf wrote:So, today I've been working on a new version that includes an optional "Required String" parameter. It doesn't calculate things faster, but it shortens the list of results, by not including any which don't contain the specified string within. I decided to do that after spending a while yesterday searching thoughout a list of results for those who contained a white space (and ignoring the rest). I figured out that instead of having to manually filter the results, it would be smarter to have the program be able to do it by itself in advance...

I'll be uploading it in the next days, after I finish tweaking it and making all 4 available versions of it like for the previous ones...
So, it took a while, but today I finally went back to finish cleaning up the code and generating the alternate versions, and just a few minutes ago I've already uploaded all 4... look for version 28 and 29 in the Tools section:

https://www.trescom.org/tools/

And all these recent developments by LtSten have made me consider adding into versions 30 and 31 the ability to use dictionaries (word lists) for the Leading, Trailing and Middle strings, instead of manually entering just one for each run of the program... of course, that will mean longer computing times if those options are selected.

Other features I had been considering adding (prior to these last ideas): a button to automatically open the results list, and adding a header to the results detailing the parameters selected at runtime.
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files
User avatar
Draconisaurus
T-Rex Killer
T-Rex Killer
Posts: 14032
Joined: Mon Dec 06, 2004 5:21 pm
Antispam: No
Location: Isla Sorna
Contact:

Re: Trespasser - CRC-32 hash tools

Post by Draconisaurus »

"GUN - HAPPINESS-PROSPECTS-01". Oye! Those lists are chalk full of amusing strings. It's quite funny when they randomly make sense. Oh, well, I've looked at them too long, making my head spin...
User avatar
machf
T-Rex Killer
T-Rex Killer
Posts: 12368
Joined: Thu Apr 24, 2003 11:20 pm
Location: Lima, Peru
Contact:

Re: Trespasser - CRC-32 hash tools

Post by machf »

So, to give you an idea of what the "backwards" versions can be used for, here are some listings I generated (the longest ones, for 8 characters, took about an hour) for the entry in the early builds' Ambient.tpa files with the hash value BE298162:

http://www.tresocm.org/files/crackcrc8char.zip

http://www.tresocm.org/files/crackcrc2-8char.zip

Each set contains 3 results files, for 6, 7 and 8 random characters, the first set having been calculated assuming the name begins with the leading string "AMB-" and the second set without any leading string...

Feel free to browse through the results to see if you find anything that makes any sense... the 6-character ones are ~100KB (for almost 9000 results), the 7-character ones are bout ~3MB (almost 270K results), and the 8-character ones are over ~100MB each (over 8 million results).

And why use the backwards order? Because this way, for example, all the results ending in "-01" are grouped together ( or in "-02", or whatever... now I'm thinking maybe I should have also included results ending in " A" and " B").
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files
User avatar
tatu
-=TresCom Website Manager=-
-=TresCom Website Manager=-
Posts: 5087
Joined: Fri Jun 24, 2005 9:40 pm
Antispam: No
Location: Sweden
Contact:

Re: Trespasser - CRC-32 hash tools

Post by tatu »

machf wrote:So, to give you an idea of what the "backwards" versions can be used for, here are some listings I generated (the longest ones, for 8 characters, took about an hour) for the entry in the early builds' Ambient.tpa files with the hash value BE298162:

http:///www.tresocm.org/files/crackcrc8char.zip

http:///www.tresocm.org/files/crackcrc2-8char.zip

Each set contains 3 results files, for 6, 7 and 8 random characters, the first set having been calculated assuming the name begins with the leading string "AMB-" and the second set without any leading string...

Feel free to browse through the results to see if you find anything that makes any sense... the 6-character ones are ~100KB (for almost 9000 results), the 7-character ones are bout ~3MB (almost 270K results), and the 8-character ones are over ~100MB each (over 8 million results).

And why use the backwards order? Because this way, for example, all the results ending in "-01" are grouped together ( or in "-02", or whatever... now I'm thinking maybe I should have also included results ending in " A" and " B").
The links are misspelled ;)

I think including A and B would've been good.
Question, as BE298162 and the one after 2720D0D8 should have the same ID, except for the final number. Could generating names for both, add them both into a txt and sort them alphabetically help with finding them? That way IDs that do not have both 01 and 02 (for example)

Not sure including A and B would help much. Ambients don't appear to end on that without a number after. But I don't know.

Question: As "BE298162" and "2720D0D8" have the same ID except for the last number/letter, would it help to generate files for both, combine the result and sort them in alphabetical order? That way we could remove any ID that does not end in 1/2, A/B can be removed?

Also, are we sure they end in same sequence (01/02)? When I look at the CRCrev for the other ones we have:

Code: Select all

AMB1	616D6231
AMB2	616D6232
AMB4	616D6234
AMB5	616D6235
AMB7	616D6237
AMB1 ends in 21, AMB2 in 32, AMB4 in 34 and therefore AMB3 would've ended in 33.

Code: Select all

886C1684	 (possibly AMB- +1)
886C1687	 (possibly AMB- +2)
If first is 01, shouldn't the second be 04 as there is no ending in 85 and 86?
Active project: Trespasser: Isla Sorna
Status:
BE-PH1: Released
PH2-IT: Pre-released
PL-SUM: In production

"...there used to be more benches, but InGen's workers removed them during the evacuation in the name of framerate."
User avatar
machf
T-Rex Killer
T-Rex Killer
Posts: 12368
Joined: Thu Apr 24, 2003 11:20 pm
Location: Lima, Peru
Contact:

Re: Trespasser - CRC-32 hash tools

Post by machf »

tatu wrote:
The links are misspelled ;)
Ugh. Fixed them.
I think including A and B would've been good.
Question, as BE298162 and the one after 2720D0D8 should have the same ID, except for the final number. Could generating names for both, add them both into a txt and sort them alphabetically help with finding them? That way IDs that do not have both 01 and 02 (for example)
Since not necessarily the 1st one ends in 01 and the second in 02 (meaning, they could be in the reverse order), it's better to check both possibilities for the first one and see which one it is, then the second one will automatically have to be the other...
Not sure including A and B would help much. Ambients don't appear to end on that without a number after. But I don't know.

Question: As "BE298162" and "2720D0D8" have the same ID except for the last number/letter, would it help to generate files for both, combine the result and sort them in alphabetical order? That way we could remove any ID that does not end in 1/2, A/B can be removed?
It wouldn't. if you have two strings which only differ by 1 character (and you know which one), it's a waste of time to calcualte strings for both of them. You only do one and then you know the other from it. Same as with those 13 names, there was a group of 5, a group of 6 and a group of 2... finding a single one would give all the rest in the group.
Also, are we sure they end in same sequence (01/02)? When I look at the CRCrev for the other ones we have:

Code: Select all

AMB1	616D6231
AMB2	616D6232
AMB4	616D6234
AMB5	616D6235
AMB7	616D6237
AMB1 ends in 21, AMB2 in 32, AMB4 in 34 and therefore AMB3 would've ended in 33.

Code: Select all

886C1684	 (possibly AMB- +1)
886C1687	 (possibly AMB- +2)
If first is 01, shouldn't the second be 04 as there is no ending in 85 and 86?
No. You need to look at it in binary arithmetic form.
Using the exclusive-or (XOR) operator:

1 XOR 2 = 0001 XOR 0010 = 0011 = 3 // difference between 1 and 2
2 XOR 3 = 0010 XOR 0011 = 0001 = 1 // difference between 2 and 3 (or 0 and 1)
3 XOR 4 = 0011 XOR 0100 = 0111 = 7 // difference between 3 and 4
4 XOR 7 = 0100 XOR 0111 = 0011 = 3 // difference between 1 and 2 (or 5 and 6)
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files
User avatar
tatu
-=TresCom Website Manager=-
-=TresCom Website Manager=-
Posts: 5087
Joined: Fri Jun 24, 2005 9:40 pm
Antispam: No
Location: Sweden
Contact:

Re: Trespasser - CRC-32 hash tools

Post by tatu »

I see. I am not a coding language person so I have little knowledge how stuff like this work, but thanks :)
Active project: Trespasser: Isla Sorna
Status:
BE-PH1: Released
PH2-IT: Pre-released
PL-SUM: In production

"...there used to be more benches, but InGen's workers removed them during the evacuation in the name of framerate."
User avatar
machf
T-Rex Killer
T-Rex Killer
Posts: 12368
Joined: Thu Apr 24, 2003 11:20 pm
Location: Lima, Peru
Contact:

Re: Trespasser - CRC-32 hash tools

Post by machf »

I'm glad to announce that I successfully got the optional header working in my next version of this tool. Now, let's try to add the other functions...
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files
User avatar
LtSten
Gallimimus
Gallimimus
Posts: 609
Joined: Fri Nov 27, 2009 8:28 pm
Antispam: No
Location: England, UK

Re: Trespasser - CRC-32 hash tools

Post by LtSten »

Right, I've finally added some proper command line arguments to my tool (and done a little bit of optimisation). I've also written a couple of scripts to extract updated word lists and CRC values, rather than doing it manually each time. All the fun stuff is in the attached zip.

Firstly, there are two Python scripts (you'll need xlrd to run them, however). Both operate on all xls files in the current working directory:
  • ExtractWords.py goes through each sheet of each workbook and extracts values from the columns containing "names", and outputs the results to words.csv (which is just one entry per line) - the result of running it on all of machf's xls files as of today gives the attached words.csv.
  • ExtractCRCs.py is similar, extracting all name-CRC pairs (where the CRC is the "ID" column - that is, the one that isn't CRCrev), outputting to "crc.csv". I've included the result of this as above.
Now comes the C++ CRCFinder, which is what I've been using to find these CRCs. It's an entirely command-line tool (x64 binary included) and likely requires the latest VC++ runtime available from here. You can run "CRCFinder.exe --help" to get an idea of what the possible parameters are. An example use is

Code: Select all

CRCFinder.exe -t 8 -d TresWordsTrim.txt -c crc.csv -s "-| | - |" -e "1|01|a" -a 3
CRCFinder.exe --threads 8 --dictionary TresWordsTrim.txt --crc crc.csv --separators "-| | - |" --endings "1|01|a" --appendcount 3
where I've also given the somewhat more understandable long-form version, equivalent to the first, and you should choose the thread count to match the number of threads you have available (or want to use).
As a minimum, you should specify:
  • -d / --dictionary, the text file containing one word per line, each of which is used as an allowable word (these should be lowercase)
  • -c / --crc, a CSV of crc,text pairs, one per line. Each text string equal to [unknown] is used to populate the list of CRCs to find.
  • -s / --separators, a string delimited by vertical pipes (i.e. the "|" character) of separators to place between words
  • -e / --endings, as with separators, except these are appended only at the very end (i.e. never between words)
  • -a / --appendcount, the maximum number of words to try and stick together. 2 or 3 is a sensible choice - I'd only recommend 4 with a very short word list. For four-word searches, it's often better to specify somethng like --prefix "gun - " and get three words appended to these combinations.
There's also a "--generatewordlist filename" option, which takes a name list generated by ExtractWords.py and splits them up into individual words on spaces and dashes (so the name "GUN - Pistol - Drop - Wood" would get converted into four words: "gun", "pistol", "drop", and "wood"). This is how I generated the attached TresWords.txt. This often needs a little bit of a manual pass (it tries to remove duplicates, those ending in numbers, but a few anneammophrase get through since they end in letters instead) to filter a few remaining irrelevant entries.

Running the CRC finder generates an output file (defaults to "output.csv", but you can override this with --output, and I've included the results of the running the above too) containing matches for CRCs provided in the CRC list. Usually most of these are not correct, but if you've specified a good enough word list (just the right balance between possibilities and final results), then you should have a reasonable number of results to sift through manually. If one looks plausible, it's often worth finding it in machf's spreadsheet and checking what the nearby strings are, since usually it's part of a relevant group.

If you run the above, then there's a few it finds that I think are new, too:
  • be298162,"testambient1" (present in PCGamerTPAs, E3TPAs, V55TPAs only - Ambient.tpa). 2720D0D8 is then given by "testambient2".
  • 86fa9db4, "spec-car squeak" (present in TresTPAs, DemoTPAs, and Beta 96/97/99/103)
  • f22b45c0, "spec-generator motor" (V55, Beta 96, 97, 99, 103)
372D5110 and AB624FEB sound like they pair with "spec-car squeak" - I had a very quick go but I haven't been able to crack them yet.

As well as a Trespasser word list, I've included the two other word lists I've been using: commonwords.txt, which has the 10000 most commonly used words in English, and words.txt, which is a huge list of 466550 words. commonwords.txt works best with 2 appendings, and words.txt with just 1.

Lastly, I've included the source code under the Source_CRCFinder_Standalone folder in the zip. It's a Visual Studio (2019) solution, requires reasonable C++20 support (concepts, mostly - I'm building against 16.8.3, I think you'll require at least 16.8) and I'd recommend building it in Release for now. The main bits of interest will be Source/CRCFinder/CRCFinder.cpp and Source/CommonCore/CRC32.h.
Attachments
CRCFinder.zip
(1.43 MiB) Downloaded 218 times
Bringing TPA CRCs up the well, amongst other file format adventures.
Blender 3.0 TPM Plugin [Project GitHub]
User avatar
tatu
-=TresCom Website Manager=-
-=TresCom Website Manager=-
Posts: 5087
Joined: Fri Jun 24, 2005 9:40 pm
Antispam: No
Location: Sweden
Contact:

Re: Trespasser - CRC-32 hash tools

Post by tatu »

LtSten wrote:[*] be298162,"testambient1" (present in PCGamerTPAs, E3TPAs, V55TPAs only - Ambient.tpa). 2720D0D8 is then given by "testambient2".
I think this would fit very well with that a lot of the first entries in the early TPAs are test sounds. I wonder if the unknown ID that is at the beginning in some of the Stream tpas have something similar in name, and was used to test the voiceovers/stream tpa.
Active project: Trespasser: Isla Sorna
Status:
BE-PH1: Released
PH2-IT: Pre-released
PL-SUM: In production

"...there used to be more benches, but InGen's workers removed them during the evacuation in the name of framerate."
User avatar
Draconisaurus
T-Rex Killer
T-Rex Killer
Posts: 14032
Joined: Mon Dec 06, 2004 5:21 pm
Antispam: No
Location: Isla Sorna
Contact:

Re: Trespasser - CRC-32 hash tools

Post by Draconisaurus »

Cool LtSten... Hmm. While using what I am very sure were the b96 TPAs, I was able to get "spec-car squeak" to play but not "spec-generator motor".

For the retail ones... "GUN - AUTOMATIC CLICK 01" surprises me as a gun sound. Didn't realize guns made this. That's neat. Trailer Fall01 sounds familiar to my ears from the foley but Fall02 might be new. Then the same with Squeak01 but Squeak02 sounds longer than sounds I recall hearing the trailer make. The lot of them seem useful for a procedural puzzle which threatens the fall of a trailer while the player does a sequence of things.

The two Lindstradt sounds are nice and new to my ears... tho sound almost the same as each other. Insertion01 and Insertion02 sound great and I imagine myself to have used them several times by this date if their names had been found. TptDac might have, too.

I have just tried and can't get "testambient1" to work... I went to try the hacked TresEd for easier TPA source confirmation, but the server files are being migrated. At any rate, nice sounds! May be useful to modders in future. We seem to have some around here these days.
User avatar
LtSten
Gallimimus
Gallimimus
Posts: 609
Joined: Fri Nov 27, 2009 8:28 pm
Antispam: No
Location: England, UK

Re: Trespasser - CRC-32 hash tools

Post by LtSten »

Ran "spec-car " through using commonwords.txt with 2 appendings. This gave me 372d5110 as "spec-car fall final". I then went and guessed "spec-car fall impact" for ab624feb. These definitely fit the sounds!

Update: Combined the Tres list with the common words. c7d81246 seems to be "spec-workstation boot".
8e748be0 (non-retail) could be "spec-water pipes01". I'm not very convinced though - I've checked the sound and it's not obviously correct (at least to me).

Update 2: 2720d0d8 could be "amb-large forest - test-a" from the E3, PCGamer, and V55 TPAs. This gives "amb-large forest - test-b" for be298162.
Bringing TPA CRCs up the well, amongst other file format adventures.
Blender 3.0 TPM Plugin [Project GitHub]
User avatar
machf
T-Rex Killer
T-Rex Killer
Posts: 12368
Joined: Thu Apr 24, 2003 11:20 pm
Location: Lima, Peru
Contact:

Re: Trespasser - CRC-32 hash tools

Post by machf »

LtSten wrote:Right, I've finally added some proper command line arguments to my tool (and done a little bit of optimisation). I've also written a couple of scripts to extract updated word lists and CRC values, rather than doing it manually each time. All the fun stuff is in the attached zip.

Firstly, there are two Python scripts (you'll need xlrd to run them, however). Both operate on all xls files in the current working directory:
  • ExtractWords.py goes through each sheet of each workbook and extracts values from the columns containing "names", and outputs the results to words.csv (which is just one entry per line) - the result of running it on all of machf's xls files as of today gives the attached words.csv.
  • ExtractCRCs.py is similar, extracting all name-CRC pairs (where the CRC is the "ID" column - that is, the one that isn't CRCrev), outputting to "crc.csv". I've included the result of this as above.
Now comes the C++ CRCFinder, which is what I've been using to find these CRCs. It's an entirely command-line tool (x64 binary included) and likely requires the latest VC++ runtime available from here. You can run "CRCFinder.exe --help" to get an idea of what the possible parameters are. An example use is

Code: Select all

CRCFinder.exe -t 8 -d TresWordsTrim.txt -c crc.csv -s "-| | - |" -e "1|01|a" -a 3
CRCFinder.exe --threads 8 --dictionary TresWordsTrim.txt --crc crc.csv --separators "-| | - |" --endings "1|01|a" --appendcount 3
where I've also given the somewhat more understandable long-form version, equivalent to the first, and you should choose the thread count to match the number of threads you have available (or want to use).
As a minimum, you should specify:
  • -d / --dictionary, the text file containing one word per line, each of which is used as an allowable word (these should be lowercase)
  • -c / --crc, a CSV of crc,text pairs, one per line. Each text string equal to [unknown] is used to populate the list of CRCs to find.
  • -s / --separators, a string delimited by vertical pipes (i.e. the "|" character) of separators to place between words
  • -e / --endings, as with separators, except these are appended only at the very end (i.e. never between words)
  • -a / --appendcount, the maximum number of words to try and stick together. 2 or 3 is a sensible choice - I'd only recommend 4 with a very short word list. For four-word searches, it's often better to specify somethng like --prefix "gun - " and get three words appended to these combinations.
There's also a "--generatewordlist filename" option, which takes a name list generated by ExtractWords.py and splits them up into individual words on spaces and dashes (so the name "GUN - Pistol - Drop - Wood" would get converted into four words: "gun", "pistol", "drop", and "wood"). This is how I generated the attached TresWords.txt. This often needs a little bit of a manual pass (it tries to remove duplicates, those ending in numbers, but a few anneammophrase get through since they end in letters instead) to filter a few remaining irrelevant entries.

Running the CRC finder generates an output file (defaults to "output.csv", but you can override this with --output, and I've included the results of the running the above too) containing matches for CRCs provided in the CRC list. Usually most of these are not correct, but if you've specified a good enough word list (just the right balance between possibilities and final results), then you should have a reasonable number of results to sift through manually. If one looks plausible, it's often worth finding it in machf's spreadsheet and checking what the nearby strings are, since usually it's part of a relevant group.
Goung to look at all that later. Thanks!
If you run the above, then there's a few it finds that I think are new, too:
  • be298162,"testambient1" (present in PCGamerTPAs, E3TPAs, V55TPAs only - Ambient.tpa). 2720D0D8 is then given by "testambient2".
  • 86fa9db4, "spec-car squeak" (present in TresTPAs, DemoTPAs, and Beta 96/97/99/103)
  • f22b45c0, "spec-generator motor" (V55, Beta 96, 97, 99, 103)
So... they used "car" instead of "vehicle". I should have gueesed it. It's even simpler.
"Testambient" makes a lot of sense... AND it's also buried among my results for 8 random characters (crackcrc2-8char.txt), if you search for it. It's just that that list is SO long I didn't get to go through all of it yet...
"Generator"... hmmm.. I always thought it was some othe rpiece of the GroThermal Plant...
372D5110 and AB624FEB sound like they pair with "spec-car squeak" - I had a very quick go but I haven't been able to crack them yet.
Well, using as a reference the "trailer fall" and "trailer squeak", my guess is that they're some variation of "car fall", then...
(And indeed, judging by your later post, I see so it was)
As well as a Trespasser word list, I've included the two other word lists I've been using: commonwords.txt, which has the 10000 most commonly used words in English, and words.txt, which is a huge list of 466550 words. commonwords.txt works best with 2 appendings, and words.txt with just 1.

Lastly, I've included the source code under the Source_CRCFinder_Standalone folder in the zip. It's a Visual Studio (2019) solution, requires reasonable C++20 support (concepts, mostly - I'm building against 16.8.3, I think you'll require at least 16.8) and I'd recommend building it in Release for now. The main bits of interest will be Source/CRCFinder/CRCFinder.cpp and Source/CommonCore/CRC32.h.
Going to look at that, as I said.
LtSten wrote:Ran "spec-car " through using commonwords.txt with 2 appendings. This gave me 372d5110 as "spec-car fall final". I then went and guessed "spec-car fall impact" for ab624feb. These definitely fit the sounds!
See? They follow the same pattern...

Update: Combined the Tres list with the common words. c7d81246 seems to be "spec-workstation boot".
8e748be0 (non-retail) could be "spec-water pipes01". I'm not very convinced though - I've checked the sound and it's not obviously correct (at least to me).
Ah, well, "workstation" coming after steam, makes sense, and it WAS something starting,
As for "water pipes"... I'll listen to the sound. It does fit with the previous ones, though...
EDIT: well, it's steaming hot water, after all. Doesn't sound like regular cold water. I'm including it as the name is too fitting not to be it. And that was one of the parts of the GeoThermal Plant for which I was looking for a possible sound... but I was trying things like "production well" (as opposed to "re-injection well").

Update 2: 2720d0d8 could be "amb-large forest - test-a" from the E3, PCGamer, and V55 TPAs. This gives "amb-large forest - test-b" for be298162.
Nah, those are most likely the previous ones, "testambient1" and "testambient2" as the earlier builds have several names like that...
Visit The Carnivores Saga - a forum devoted to modding Action Forms' Carnivores, Carnivores 2 and Carnivores: Ice Age games
Tres WIP: updated T-Script Reference and File Formats documents
Sound name listings for the Demo (build 117), Retail (build 116), Beta 103, Beta 99, Beta 97, Beta 96, Build 55, PC Gamer Alpha (build 32) and E3 1998 Alpha (build 22) TPA files
Post Reply