Recover video recordings from Panasonic DVR based on MEIHDFS, BDAV, and RPLS

Therefore, go home to your villages and repair the IT of your families!

(Jesus, never)

Despite all circumstances, I managed to visit my family this Christmas. And to some extend, it is tradition that I use some of the time at home to repair/update/replace some IT devices of my family.

This year however, I had a special challenge: Copy the recordings of the family camcorder, that were transferred to the external HDD of a Panasonic DVR to a usual PC.

Sounds easy, doesn’t it?

I was warned, because the “straight forward” solution, of attaching the external USB HDD of the DVR to a PC/Laptop didn’t work. So I was expecting a nice personal CTF-style Christmas challenge refreshing some of the forensics knowledge I learned at my university. I wasn’t disappointed.

Background and Setup

So, my family has an Panasonic Blu-Ray Recorder DMR-BST820 with an external USB HDD attached. For over 20 years, we used (multiple versions) of a MiniDV camcorder (i.e. some kind of video on tape) to record family events. Recently, we had discussed to “digitize” this old tape-based recordings to safely store them on our internal NAS (with RAID, of course).

After trying different approaches, the best way was to connect the camcorder via S-Video to the named DVR (instead of e.g. trying to digitize VHS versions of the recordings). The DVR has the option to attach external HDDs. So, the plan was to copy the videos from the old MiniDV tapes via the camcorder and S-Video to this new external HDD of the DVR. Afterwards, we could copy the recordings to the NAS.

Some month later, my grandfather has copied all (nearly hundred) of MiniDV videos to the external HDD, as described.

But then there was the nasty surprise: No PC/Laptop could read this external HDD (my family did try Linux by themselves)!

Problem 1: A custom file system

On attaching the external HDD (the one on which the Panasonic DVR stored the recordings), dmesg showed the following:

....
 usb 3-2: new SuperSpeed Gen 1 USB device number 3 using xhci_hcd
 usb 3-2: New USB device found, idVendor=1058, idProduct=2600, bcdDevice=20.06
 usb 3-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
 usb 3-2: Product: Authorized HDD
 usb 3-2: Manufacturer: Western Digital
 usb 3-2: SerialNumber: 574343344D35584E5854395A
 usb-storage 3-2:1.0: USB Mass Storage device detected
 scsi host1: usb-storage 3-2:1.0
 scsi 1:0:0:0: Direct-Access     WD       Authorized HDD   1    PQ: 0 ANSI: 6
 sd 1:0:0:0: Attached scsi generic sg0 type 0
 sd 1:0:0:0: [sdb] Spinning up disk...
 .........ready
 sd 1:0:0:0: [sdb] 3907028992 512-byte logical blocks: (2.00 TB/1.82 TiB)
 sd 1:0:0:0: [sdb] Write Protect is off
 sd 1:0:0:0: [sdb] Mode Sense: 47 00 10 08
 sd 1:0:0:0: [sdb] No Caching mode page found
 sd 1:0:0:0: [sdb] Assuming drive cache: write through
 sd 1:0:0:0: [sdb] Attached SCSI disk
 FAT-fs (sdb): bogus number of reserved sectors
 FAT-fs (sdb): Can't find a valid FAT filesystem
 FAT-fs (sdb): bogus number of reserved sectors
 FAT-fs (sdb): Can't find a valid FAT filesystem
 FAT-fs (sdb): bogus number of reserved sectors
 FAT-fs (sdb): Can't find a valid FAT filesystem

 ufs: You didn't specify the type of your ufs filesystem
 
 mount -t ufs -o ufstype=sun|sunx86|44bsd|ufs2|5xbsd|old|hp|nextstep|nextstep-cd|openstep ...
 
 >>>WARNING<<< Wrong ufstype may corrupt your filesystem, default is ufstype=old
 ufs: ufs_fill_super(): bad magic number
 ufs: ufs_fill_super(): bad magic number
....

Some of the FAT-fs/ufs error messages did occur automatically, some where triggered by myself while trying different (non-usual) file systems. In summary, no one (i.e. FAT-X, ext-X, ufs, exfat, etc.) did work. Also the handy tool TestDisk wasn’t of any help.

Soon, I figured out that the problem wasn’t with the USB drive (as I did guess based on the initial error messages) or the operating system failing to detect a file system, but that the problem is rather with the file system itself. Consequently, I started to concentrate my search on "recover panasonic recordings". Shortly before I would have used foremost (what wasn’t very promising, but is always an option), I discovered a very helpful tool: panasonic-rec!

MEIHDFS: The used file system

After some research, I figured out that Panasonic (and other “big names” in the entertainment industry) is using a properitary file system for the external HDD: In this case MEIHDFS. I still don’t know why, because the side effect of this is that a once “initialized” HDD could only be used by the very same DVR device. AFAIK also if the DVR brakes and one buys the exact same device again, it can not read the old external HDD…WTF.

Further information to MEIHDFS and how leecher1337 and hkmaly did reverse it, can be found here (very interesting to read).

But thanks to the mentioned tool, I had a way to dump the data from the HDD.

Solution:

Preparation:

$ git clone https://github.com/leecher1337/panasonic-rec.git
$ cd panasonic-rec/meihdfs/extract/
$ make
# there I had a very strange experience: make did work on first try without any errors!

As it is described in the README.txt the data can be extracted by executing:

$ ./extract_meihdfs /dev/sdX /path/to/output/folder

The output folder must have enough space to store all contents of the external HDD, there is no further way to filter or select the content.

This worked seamlessly!

But, in contrast to the description in the readme of panasonic-rec, the resulting files had a different file structure:

target_folder
└── BDAV
    ├── CLIPINF
    ├── PLAYLIST
    └── STREAM

So, here I couldn’t continue with this tool.

Problem 2: A strange video format

Ok, it was easy to figure out that the videos were in BDAV/STREAM, but with strange formats: .m2ts and .m2ps. It turned out that is is some kind of MEPG-2 Transport Stream and that ffmpeg can convert it.

So, using a short script, I converted the video files to “boring” h264/mp4:

#!/bin/bash

RED='\033[0;31m'
NC='\033[0m' # No Color
GREEN='\033[0;32m'

source="/path/to/BDAV/STREAM"
target="/path/to/another/dir/"

for file in $source/*.m2?s; do
  name=${file##*/}
  base=${name%.m2?s}
  echo -e "${GREEN} ffmpeg -i $file -vcodec libx264 -preset veryfast -crf 20 -acodec ac3 -vf "yadif" $target/$base.mp4 ${NC}"
  ffmpeg -i $file -vcodec libx264 -preset veryfast -crf 20 -acodec ac3 -vf "yadif" $target/$base.mp4
done

Now, my default video tools (e.g. vlc) can play the videos without problems.

Problem 3: BDAV with (custom?) playlist format

So far, so good. But my grandfather had put some effort in naming all the converted videos and until now, I was just able to have files like 0001.mp4, 0002.mp4, and so on.

Therefore, I asked myself if there wouldn’t exist a way to extract those metadata as well. After some tries, I located them in the BDAV/PLAYLIST folder.

The .rpls format

RPLS is used by some vendors for the BDAV (Blu-ray Disc Audio/Visual) format of Blu-Ray discs to store the metadata of the recorded videos.

BDAV is intended to be used by consumers as Blu-ray Disc file structure to safe recorded videos, including from personal camcorders. But it is also used by some DVR/AVR vendors (e.g. Panasonic) as organization schema for external HDDs.

As shown above, the BDAV file structure is as follows:

BDAV/
├── info.bdav
├── menu.something
├── CLIPINF
│   ├── 00001.clpi
│   ├── 00002.clpi
│   ├── 00003.clpi
│   ├── .....
├── PLAYLIST
│   ├── 00001.rpls
│   ├── 00002.rpls
│   ├── 00003.rpls
│   ├── .....
└── STREAM
    ├── 00001.m2ts
    ├── 00002.m2ps
    ├── 00003.m2ps
    ├── .....

• PLAYLIST contains the metadata of the videos
• STREAM contains the videos
• (CLIPINF should contain some scene information…didn’t help me a lot)

Next, I figured out that the file names do not match! I.e. it is not required that 0001.rpls belongs to 0001.m2ps!

However, the .rpls file contains the name of the STREAM file that it describes. Hence, a short script can extract the name (and other metadata) of the .rpls files and match it to the video files.

Sadly, I haven’t found a lot of information regarding the .rpls file structure, but xxd did help a lot (as usual).

For example:

$ xxd 0042.rpls
00000000: 504c 5354 3031 3030 0000 0728 0000 0748  PLST0100...(...H
00000010: 0000 07ac 0000 060c 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 05de 0000 1102  ................
00000030: 0200 2042 1123 1534 2800 3431 0103 01a0  .. A.#.4(.41....
00000040: 0001 0003 4156 3100 0000 0000 0000 0000  ....AV1.........
00000050: 0000 0000 0000 0000 1453 4f4d 4520 5641  .........SOME VA
00000060: 4c55 4142 4c45 2054 4954 4c45 2000 0000  LUABLE TITLE ...
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................
....
00000710: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000720: 0000 0000 0000 0000 0000 001c 0001 0001  ................
00000730: 0000 0014 3030 3030 324d 3250 5300 0100  ....00002M2PS...
00000740: 0000 2e18 063b b4f8 0000 0060 0002 0318  .....:.....`....
00000750: 0103 0000 0009 33dc ffff ffff 0000 0000  ......3.........
....

So, here the title SOME VALUABLE TITLE has the length of 0x14 and belongs to the file 00002.M2PS.

Consequently, the file structure appeared to me as follows:

I’ve also found a german blog entry with some more file structure information, but only parts of the reported structure were valid in my case.

Solution

With that information, I wrote a short python script (available on github) to extract the metadata I was interested in.

The script extracts the title of the playlist file, matches it to the corresponding video file (i.e. extracting the file name), stores the title as metadata in the video file, and finally renames/copies the video file.

Usage:

$ python3.8 main.py /path/to/BDAV/PLAYLIST /path/to/BDAV/STREAM/or-other-video-folder /path/to/processed-and-renamed-output-dir

(For further details, please see the repo).

Of course, I could have converted the .m2ts files in this step as well, since I have to use ffmpeg to append the metadata to the video. But afterwards one is always wiser.

In Conclusion…

Finally, I have usable .mp4 files that have a meaningful file name on the family NAS!

Haven’t touched things like “unusual” file-systems, file carving, and reverse engineering for some time, I had a little bit of fun and a good change of refreshing some old and learning a few new forensic tricks. In the end, the old tape camcorder recordings of family events got digitized and safely stored, eventually.