WORKAROUND fixing incomplete "Recordinglist" in webinterface

[Langhuse]

I was missing 24 out of 72 recordings in the recordinglist (Webinterface-> Recordings)
- Using the task -> "Refresh Recording DB" did not help.
- stopping recording service and delete all databases in "DBViewer->Database" using windows explorer did not help.



One problem is a "fishy" traversal of the recording sub-directories when refreshing the recordinglist.

There is also a problem with some "cashed" information used even after a stop server followed by a "Refresh Recording DB"


To illustrate this I will document what I had to do to recover all the missing entries in my Recordinglist

Setup:
- WIN7
- lastet Recordingservice 1.29.0.0
- recordings held on a NAS samba drive (refered using UNC path)
- I had earlier moved the recording to a NAS samba drive, but even more recent recording had problems


FIX-1: update Recording Service to include all sub-directories with recordings

The issue here is that it is not enough to include the top-level directory for the recordings.

This is a mystery as some of the recordings on some subdirectories were found.

What I did was
1) add all the sub-directories using
Recording Service -> configure -> Recorder
and adding all subdirs to the "Directory" list
2) "Refresh Recording DB"

This did recover approx 15 entries so they were now visible in the recordinglist.

 

As far as I recall it was not neccesary to delete all databases in the Database directory.

FIX-2: rename .ts/.log/.txt - adding a space in the filename

To fix the remaining files I had to rename the files.

It was not enough to
- stop the server
- delete the content of "Database"
- start server
- "Refresh Recording DB"

Why a single space in the filenames should help is unclear. But maybe something is cached somewhere ??

I renamed the files adding a single space before the .ts/.log/.txt file extension.
(very simple in win7: select the 3 files; F2; left ; right; insert one " " ; return)

This did recover the remaining entries so they were visible in the recordinglist.


"refresh DB" does not refresh information from the .txt file

While debugging the above i did all kind of things.

One test I made was
- Modify the Recording description in the "Description=" field inside the .txt file (checking for suspicious chars)
- "refresh DB"

but the recording description (when mouse hovering over title in the recording list) was still showing the old text.

- stop server
- modify .txt
- start server
- "refresh DB"

did not work either

- stop server
- modify .txt
- delete all databases in "Database"
- start server
- "refresh DB"

Did work, so refresh DB is probably not testing correctly for changes in all fields.

Personally I would prefer a "reBUILD Recording DB" instead of a "refresh". This would make a clean build, instead of having to add additional/new code trying to refresh only changed fields.

This is additional complex code not required because working code was already there to build the very first complete version - why not reuse?


Tools used while debugging

I used cygwin on win7.

 

Concept: I checked the filenames inserted into the SvcDatabase.db3 against the files found in the filesystem and keept going until they were identical

I used the Makefile below (for your inspiration)

So the job was completed successfully when the database matched the recordings in the filesystem

 

Unsuccessfull run:

make

Unique filenames in DB or FS (indented)

comm -3 files.inDB.lst files.inFS.lst
\\NAS2\share\OptagetTV\Other\Kun 5 ingredienser! (3)_2012-12-09_13-30-13_TNTT.ts
...

...

...

\\NAS2\share\OptagetTV\Save\randi til dans\Thomas Skovs Sportsprogram 2.0 - Linedance (6 10)_2013-09-24_21-01-10_DR3.ts
Missing files in DB: 25
Matched files in DB: 45

 

Successfull run (some hours later)

make all

Unique filenames in DB or FS (indented)

comm -3 files.inDB.lst files.inFS.lst
Missing files in DB: 0
Matched files in DB: 70



---------- Makefile ---------------

all: get gen compareFileFound

get:
cp /cygdrive/c/ProgramData/CMUV/DVBViewer/Database/*db3 .

gen:
echo '.dump' | sqlite3.exe Logos.db3 > Logos.db3.lst
echo '.dump' | sqlite3.exe MediaDatabase.db3 > MediaDatabase.db3.lst
echo '.dump' | sqlite3.exe MusicDatabase.db3 > MusicDatabase.db3.lst
echo '.dump' | sqlite3.exe SvcDatabase.db3 > SvcDatabase.db3.lst
echo '.dump' | sqlite3.exe svcmediaAudio.db3 > svcmediaAudio.db3.lst
echo '.dump' | sqlite3.exe svcmediaPhoto.db3 > svcmediaPhoto.db3.lst
echo '.dump' | sqlite3.exe svcmediaVideo.db3 > svcmediaVideo.db3.lst
echo '.dump' | sqlite3.exe svcSettings.db3 > svcSettings.db3.lst
echo '.dump' | sqlite3.exe TimerDatabase.db3 > TimerDatabase.db3.lst
echo '.dump' | sqlite3.exe TVDatabase.db3 > TVDatabase.db3.lst


getFoundFromDB:
# extract filenames from DB and remove single qoutes "'"
grep 'INSERT INTO "recordings"' SvcDatabase.db3.lst | cut -d, -f2 | sed "s/'//g" | sort | tee >files.inDB.lst

getFoundFromFS:
# get filenames from filesystem, change "/" to "\"
find `cygpath -u //NAS2/share/OptagetTV/` | egrep '.ts$$' | sed 's|/|\\|g' | sort | tee >files.inFS.lst

compareFileFound: getFoundFromDB getFoundFromFS
@echo
@echo
@echo "Unique filenames in DataBase or FileSystem (FS indented)"
@echo
comm -3 files.inDB.lst files.inFS.lst
@echo "Missing files in DB:" `comm -3 files.inDB.lst files.inFS.lst | wc -l`
@echo "Matched files in DB:" `comm -12 files.inDB.lst files.inFS.lst | wc -l`