.cue and .sfv info/VGM archival

[CaptainCommando]

I am proposing that the VGMdb include disc gap information such as .cue files and lossless checksum files such as .sfv. While it's great we have high-res scans of many of the albums and album booklets on the site, I just don't think this would be enough information. This metadata is important for game music archival as it allows for the recreation of the original album.

I have been doing some research into the archival of CDs and believe .bin/.cue information is important for the complete archival of an album, as it would allow the original disc to be recreated exactly and would help bring the audio back into context. In addition, the files would contain complete information on the track length and disc metadata. Hydrogen Audio has an FAQ with more info on cue sheets, and the best formats here seem to be multiple files with corrected gaps and multiple files with gaps left out (though this would depend on the album, as some might contain info in the gaps):

http://wiki.hydrogenaudio.org/index.php?title=Cuesheet

A checksum file such as an .sfv would also be important to archival purposes as it would allow users who have lossless rips of game albums to verify the accuracy of their files.

One guy I know also creates a read-only RAR file containing all the tracks and disc information. The RAR format is used as it contains parity information so the RAR could be restored if part of it got corrupted.

In addition, I think it might be useful to establish some sort of standards system for collectors' digital archives, such as which lossless file format to use for backup and how track gap information should be saved. In this manner, not only would new collectors have an established system for creating and verifying archive-quality backups, but individual collectors could share consistent data while checking the accuracy of the data they already own. Historically, this would be important for future archive work.

While it's currently not possible to legally create a public VGM audio database, I think these standards would be important for when it does become possible to do so. Because the VGMdb is the best database of album information out there, I think it's important that the site contain additional metadata for archives.

As the Top 500 VGM Albums of All Time project is coming to a close, I propose that these 500 albums be prioritized for archival in this fashion.

What are your thoughts on this? What techniques do you use to create lossless backups of your own albums?

[Liontamer]

Sounds legally dicey, frankly. :-D

[CaptainCommando]

Well, a .cue file is just metadata - it just tells you what the disc contains. It's not hosting any actual data from the disc, just where it's located. So I don't see how there could be anything illegal about that. It's probably less legal to post images of album booklets.

A tutorial for backing up could be worded in such a way that it's for personal archival or portability of discs and verifying your backups rather than for sharing data.

[seanne]

I'm not really sure what you're suggesting, but I don't see what improvement hosting cue sheets would offer. They just list stuff like track length and track times, no?

[Secret Squirrel]

I can't see much use for the .sfv file other than verifying downloaded discs, which isn't something we want to promote here.

[Gigablah]

I don't think we should start hosting cuesheets, but there's the possibility of storing the metadata which allows people to reconstruct cuesheets. As in, adding options to the tracklist dropdown to download the tracklist in .txt, .csv or .cue format (all of which are dynamically generated), and so on.

[Datschge]

Since cue sheets includes centiseconds that would be of higher resolution than the current track times to employing it would be an improvement for that alone already (for the @MIDI releases I had to round most track times as I was referring to the output of the CDs directly). =P
Also (somewhat unrelated) someone mentioned using vgmdb as alternative to cddb before I'm pretty sure to achieve that the exact track times play a role there to identify CDs' total times (unless the standard actually is to cut off the centiseconds of each track and don't bother with any rounding, dunno).

[layzee]

Might this be related to the topic and be of any use?

Basically their goal is to dump/archive CD-based media properly with no data unaccounted for and for that to be the standard.

An example: Dracula ~Music Collection~, the bonus music CD that came with first-print copies of Akumajou Dracula X: Gekka no Yasoukyoku.

[CaptainCommando]

Since cue sheets includes centiseconds that would be of higher resolution than the current track times to employing it would be an improvement for that alone already

Yeah, this was what I was getting at. Cue sheets have more accurate data. A tracklist that rounds to seconds is alright if you're simply browsing, but for accurate information about an album, you need to have it down to as high a resolution as possible. These could also be used as a reliable VGM CD database to connect via an EAC plugin or something (thanks for bringing that up). I don't think it's impossible for vgmdb to do that, which is why I suggested it.

I can't see much use for the .sfv file other than verifying downloaded discs, which isn't something we want to promote here.

Ok. I certainly wasn't suggesting vgmdb promote downloading albums. I thought it would be important for album verification.

Basically their goal is to dump/archive CD-based media properly with no data unaccounted for and for that to be the standard.

Thanks for the info on redump. I wasn't aware they existed (and I'm surprised nobody mentioned it to me). I know TOSEC has a lot of information, but I don't know how they gathered it. Archival standards is something I'm interested in, thanks to my work in digital media preservation. VGMdb is a database, not an archive. But that doesn't mean it can't contain information that would be important to an archive. Anyway, this would be similar to what I was talking about. I'll run this by the SIG I'm working with and see what they think about it.

Incidentally, this site has another tutorial. The standard one guy I know uses is to use EAC to get the tracklist information. If the album is CD+G, he'll use CDRWIN.

BTW, how many VGM albums do we see with CD+G or other alternate formats and how are these documented on the site? I know the Midi Power Pro discs came with floppies containing midi files, but have never seen an actual album that contained data other than music.

[seanne]

Originally Posted by Datschge

Since cue sheets includes centiseconds that would be of higher resolution than the current track times to employing it would be an improvement for that alone already

Seriously though, who cares about centiseconds? Maybe I'm just missing the point, but when I'm looking for info about an album the tracktimes are a pretty negligible fact to begin with. So I don't see how this would be big enough an improvement to warrant all the extra work (which would be quite a fair bit).

Not to mention that, since a cue sheet can easily be tampered with, staff verification of cue's could become quite problematic. That's also why I just don't see them holding any particular authority.

[Gigablah]

Now we're getting somewhere!

Regarding cddb/freedb, VGMdb will probably just emulate its functionality; meaning that there's no need to calculate the disc hash from the individual tracktimes since we'll just store the hash itself (which we can obtain online sources) in the database.

However, higher resolution track times is a compelling argument, and I can envision an interface that allows users to submit cue sheets and parse the data inside. There's probably some other metadata regarding the physical media that we can store too.

[Gigablah]

Originally Posted by seanne

Seriously though, who cares about centiseconds? Maybe I'm just missing the point, but when I'm looking for info about an album the tracktimes are a pretty negligible fact to begin with.

You could just as well argue that some people don't care about catalogue numbers, for instance. There are sites that record tracktimes to frame level (this for example), and if we ever do it we'll most certainly be making it completely optional.

[seanne]

Originally Posted by Gigablah

You could just as well argue that some people don't care about catalogue numbers, for instance. There are sites that record tracktimes to frame level (this for example), and if we ever do it we'll most certainly be making it completely optional.

But cat. numbers do serve a purpose, after all. I just don't see how a certain track being "3:56:23" instead of "3:56" is going to be of any interest to anyone. Not that I particularly object to this idea, I just find it rather pointless. Not to mention it's could be a verification nightmare.

[Gigablah]

The purpose of a database is to catalogue data, and the more accurate and detailed the data, the better. We're planning on having pages dedicated to individual tracks down the line, so any information that goes beyond the basics is a definite plus.

You'll need the original media to reliably verify such information, naturally -- which is no different from much of the information we already submit.

[Myrkul]

Until actions taken do not promote piracy at all, i have nothing vs. it, at least sfv files should be avoided.

[Carl]

Originally Posted by CaptainCommando

BTW, how many VGM albums do we see with CD+G or other alternate formats and how are these documented on the site? I know the Midi Power Pro discs came with floppies containing midi files, but have never seen an actual album that contained data other than music.

VGCD-0149 which just came out has music + a data track full of .pdf sheet music. The Yu Suzuki Produce series of Sega albums are all CD Plus which have a compressed video clip on the disc. Those are just off the top of my head, but I think there's probably a few more too.

[Gigablah]

There's actually a lot of these albums. Do a search for "Data Track" in tracktitles and you get this list:
http://vgmdb.net/db/albums-search.ph...sults&id=15583
(Data tracks are usually dropped from the tracklisting altogether, so this is just a partial sample).

There's also albums like this: http://vgmdb.net/album/6151
which actually stuff the majority of the tracks as digital audio into the data portion. That entry also highlights the current inadequacies of the database structure.

[Echo]

Originally Posted by Gigablah

Regarding cddb/freedb, VGMdb will probably just emulate its functionality; meaning that there's no need to calculate the disc hash from the individual tracktimes since we'll just store the hash itself (which we can obtain online sources) in the database.

I think it would be awesome if VGMdb could be used to tag files in foobar, like freedb can be used now for "normal" music (because freedb sucks when it comes to vgm). It would tag the artists, arrangers, composers, track names etc. automatically. I always rip my CDs and tag them according to the information on VGMdb, but tagging albums with many artists/arrangers is a bothersome task. Last.fm users would also benefit from it, because then people would have the same tags universally.

[Kaleb.G]

I already voted it up on the feedback page, but I would love to simply draw data straight from VGMdb when using EAC (for example).