Jump to content

Unicode problem on recorded file


Recommended Posts

I've installed RS 1.7.1 and it works great but with some minor unicode issue.

 

Under recordings tab, the recordings showed properly but the file name isn't in unicode. It is not able to determine file size and also perform streaming. This apply to all browser that I used (FF, Chrome, IE8) and also iphone UI.

post-63015-0-34179600-1301810409_thumb.jpg

 

 

In fact, this happens from earlier version and I'm not sure this even works well before or not.

 

Other than this issue, everything that RS works like a charm.

 

Regards

support.zip

Link to comment

I tried to reproduce it and couldn't find any problem. Does this happen with all the recordings?

 

Yes, it did.

I've tried 3 computers, XP & also win7. Clean installation also the same.

 

And my PCs are all Traditonal Chinese.

 

And if I add the recording folder under in library, it can be display properly.

 

FYI, I browse the SQLlite database, the filename field showed incorrect file name as well.

 

post-63015-0-68240500-1302444389_thumb.jpg

Edited by hk_ben
Link to comment

Hmm. Could you please create a recording with write info file activated and upload the infofile? (it's the file with the same name as the recording but with a txt extension).

 

BTW have you tried the 1.7.1.1 version?

Link to comment

thank you but it seems that didn't work, the file names are not complete. :(

 

But I found something. it seems the (EPG) Event is not completely correct also. That very strange. Either it is all wrong or all right. This is not some external EPG, is it?

 

BTW: Does it work if you record it with the DVBViewer directly?

Link to comment

The file name is something like below, maybe the forum doesn't support unicode file name.

 

TVB-HD-Jade_森巴睇樓團_2011-04-11_11-59-36.log

----content----

TVB-HD-Jade 11/04/2011

 

\\Ts219p\TV-Recording\森巴睇樓團\TVB-HD-Jade_森巴睇樓團_2011-04-11_11-59-36.ts

 

Device: BDA 2875 TVTuner

 

11:59:36 / 00:00:00 (~ 0.0 MB) Start

11:59:38 / 00:00:01 (~ 2.3 MB) PID 851: H.264 Video, 16:9, 1920x1088, 25 fps

11:59:38 / 00:00:01 (~ 2.3 MB) PID 853: AC3 Audio 5.1, 48 khz, 384 kbps

11:59:38 / 00:00:01 (~ 2.3 MB) PID 852: AC3 Audio Stereo, 48 khz, 192 kbps

11:59:50 / 00:00:14 (~ 21.3 MB) Stop

 

Average Data Rate: 1.479 MB/s

Total Size: 21.3 MB (22340792 Bytes)

Removed H.264 Filler Data: 0.1 MB (0.5%)

----content----

 

TVB-HD-Jade_森巴睇樓團_2011-04-11_11-59-36.txt

----content----

[General]

Version=1.1

[Media]

Created=11.04.2011 11:59:36

Channel=TVB-HD-Jade

[0]

Id=23

Date=11.04.2011

Time=11:55:00

Duration=00:35:00

Title=璉桀毀???

Info=

Series=

Description=

Charset=255

Content=0

MinimumAge=0

[stats]

Errors=0

Size=21.31 MB (22340792 bytes)

Avr. Datarate=1.481 MB/s

Device=BDA 2875 TVTuner

----content----

 

 

Recorded using DVBViewer itself.

TVB-Jade_鏗鏘集_2011-04-11_19-18-20.log

----content----

TVB-Jade 11/04/2011

 

\\Ts219p\TV-Recording\鏗鏘集\TVB-Jade_鏗鏘集_2011-04-11_19-18-20.ts

 

Device: ben.itoncc

 

19:18:20 / 00:00:00 (~ 0.0 MB) Start

19:18:21 / 00:00:00 (~ 0.5 MB) PID 811: MPEG2 Video, 16:9, 720x576, 25 fps

19:18:21 / 00:00:00 (~ 0.5 MB) PID 812: AC3 Audio Stereo, 48 khz, 192 kbps

19:18:21 / 00:00:00 (~ 0.5 MB) PID 813: AC3 Audio Stereo, 48 khz, 192 kbps

19:18:34 / 00:00:14 (~ 9.6 MB) Stop

 

Average Data Rate: 0.667 MB/s

Total Size: 9.6 MB (10019460 Bytes)

----content----

 

 

TVB-Jade_鏗鏘集_2011-04-11_19-18-20.txt

----content----

[General]

Version=1.1

[Media]

Created=11.04.2011 19:18:20

Channel=TVB-Jade

LastShown=30.12.1899

LastPosition=0

Played=0

[0]

Id=30

Date=11.04.2011

Time=19:00:00

Duration=00:30:00

Title=?烾??

Info=

Description=

Charset=255

Content=0

MinimumAge=0

[stats]

Errors=0

Size=9.56 MB (10019460 bytes)

Avr. Datarate=0.668 MB/s

Device=ben.itoncc

----content----

 

 

 

 

The problem on 1.7.1.1 as well.

My recorder File Naming Scheme for both DVBViewer & Recording service.

%event\%station_%event_%year-%date_%time

Link to comment

Well it shows one thing. the EPG itself seems to be the reason. Are the EPG entries in the epg window correct? Is it internal EPG?

Link to comment

Hi MQ,

 

The link is my DVBViewer & RS EPG setting and the EPG window.EPG shows up correctly.

 

DVBViewer-EPG.JPG

 

DVBViewer-EPG-Setting.JPG

 

RS-EPG.JPG

 

RS-EPG-Setting.JPG

Edited by hk_ben
Link to comment

Tried update the Database manual.

DB-manual-update.JPG

 

Recording tab can showed the file correctly.

RS-after-manual-update-DB.JPG

 

The following field are showing garbage.

 

Field "Filename" under table "recordings" in SvcDatabe.db3

Field "Filename" under table "mediadb" in MediaDatabase.db3

Field "Filename" under table "recordings" in TVDatabase.db3

Field "Filename" under table "Video" in Svcmedia.db3

 

Unicode problem in the database ??

Edited by hk_ben
Link to comment

no more likely some other problem, otherwise we would have more reports than yours. :)

 

But I have no clue what the problem with your system might be...

Link to comment

nope, those problems were solved. :)

 

could you please zip your epg.dat and channels.dat and upload them here. Then I'll try to check where the problem lies. :)

Link to comment

Ok thank you. I "recorded" three files from different channels with chinese EPG.These are of course empty recordings, the recording service simply wrote something into it. I attached them, so you can verify the names and EPG.

 

Note: the filenames are UTF-8 encoded in the zip file. Please use use a specialized zip program (winzip, winrar, 7zip) NOT the window explorer.

 

There are no problems and no ??? in the EPG, the info files or the webinterface. I really have no clue, what is going wrong on your system... :blink:

Recorded TV.zip

Link to comment

file-encoding.JPG

 

I open the file directly from the zip, and select save as, note the difference in the encoding (UTF-8 vs ANSI).

Looks like the default encoding for the log files is having problem.

Link to comment

the log file is not of importance. It's not used anywhere, it's just a log and yes it is simple Ansi.

 

only the info (txt) file is important and this is UTF-8

Link to comment

Hm.....really weird.

 

I use the following File Naming Scheme.

%event\%station_%event_%year-%date_%time

 

Is it possible to have another varible %title (instead of the subtitle) for the naming scheme.

 

<programme start="20110417234500" stop="20110418000500" channel="985158123651089">

<eventid>17785</eventid>

<charset>255</charset>

<title><eng>HEALTH TALK<chi>從頭到腳說健康</title>

<event><eng> <chi> </event>

<description><eng> <chi> </description>

</programme>

Link to comment
  • 2 weeks later...

Hi MQ,

 

After google lot of resource, this looks like due to the issue between sqlite & windows file name. (unicode <--> ANSI/UCS-2 ?? not sure)

 

Is it possible to write the filename field in various tables directly using the file naming schema that is defined in DVB & RS (for my case, %event\%station_%event_%year-%date_%time) instead of getting it from the OS ?

 

 

 

Regards

BL

Link to comment
  • 2 weeks later...

 

nope, those problems were solved. :)

 

Only the first problem was solved. The second problem wasn't. The second problem still exists in 4.8.0.0. The second problem is very similar to Ben's. The filename field in the database table was wrong. I guess the string function used to concatenate the fields cause the problem.

post-58719-0-07617800-1304498716_thumb.jpg

Edited by cwchan3618
Link to comment

I further experiment with the "File naming scheme". I tried "%event_%year_%date_%time_%station_%event", and "%event-%year-%date-%time-%station-%event".

 

For sample 1-4, the %event expand to different value when placed at the begin and end of the file name.

For sample 4 and 5, by using "-" instead of "_" as separator, the file name was different.

 

Hope this can help you figure out the problem.

post-58719-0-35953800-1304503886_thumb.jpg

Edited by cwchan3618
Link to comment

You did it in the DVBViewer or in the recording service? cause the screenshot is a DVBViewer window.

Link to comment

You did it in the DVBViewer or in the recording service? cause the screenshot is a DVBViewer window.

 

I did it in DVBViewer.

Link to comment

what does the filenames look like in the windows explorer? do they also have wrong chars?

 

In Windows Explorer, the file name displayed correctly.

 

I think you may need the Chinese character for testing. Here are the filename in characters:

今日VIP_2011-05-04_17-43-28_高清翡翠台_今日VIP.ts

巨聲工房_2011-05-04_17-42-53_J2台_巨聲工房.ts

花漾明星_2011_05-04_17-48-11_翡翠台_花漾明星.ts

金曲挑戰站-2011-05-04-17-44-43-高清翡翠台-金曲挑戰站.ts

金曲挑戰站_2011_05-04_17-45-28_高清翡翠台_金曲挑戰站.ts

post-58719-0-09944900-1304505016_thumb.jpg

Edited by cwchan3618
Link to comment

First of all I want to thank you for your patience and help. :)

 

Could you please zip and upload the TVDatabase.db3? It's strange that all other entries are correct encoded, the filename of the file is correct only the entry of the filename in the db seems to be messed up.

 

What happens if you move the recordings to another folder, clean up and refresh the recordings db (-> options -> recordings) which will remove the entries from the db and then move the files back into the recordings folder and do a cleanup/refresh again?

Link to comment

First of all I want to thank you for your patience and help. :)

 

Could you please zip and upload the TVDatabase.db3? It's strange that all other entries are correct encoded, the filename of the file is correct only the entry of the filename in the db seems to be messed up.

Zipped file attached.

 

What happens if you move the recordings to another folder, clean up and refresh the recordings db (-> options -> recordings) which will remove the entries from the db and then move the files back into the recordings folder and do a cleanup/refresh again?

I cannot do the experiment at the moment because some recording is in progress. I'll let you know the result later.

 

Actually, I created a new subfolder called "測試" before I tried the five entries before.

TVDatabase.zip

Link to comment

ok, I think I found the problem. There was a "hidden" conversion of an UTF-8 encoded widestring to ansistring, where only ansistring should have been used.

The problem exists in the recording service and the DVBViewer.

 

You don't need to perfom the move files test anymore. Thank you again for your help. :)

 

 

And also a thank you @hk_ben, this was a really deeply hidden bug, sorry I didn't find it sooner...

Link to comment

It's great.

 

BTW, I want to report one more related problem. If the file name is in English, DVBViewer can resume playing from last exit position. However, if the file name is in Chinese, DVBViewer failed to resume playing from last exit position. After you fix the UTF-8 file name issue, please also verify the "resume playing from last exit position" feature can resume.

Link to comment

these problems are related. the filename is wrongly saved to the db but the query for the resume data works with the correctly encoded filename and doesn't find the resume data.

Link to comment
  • 3 weeks later...

RS 1.8 solved most of the issue. But still some invalid character in svcmedia.db3, table 'video' field Filename.

 

svcmedia.jpg

Edited by hk_ben
Link to comment

Tried delete the entire svcmedia.db3 and recorded a short clip...still invalid character, only for this field.

 

However, what's this db's field used for, the RS WebGUI looks good so far.

Edited by hk_ben
Link to comment

Tried empty the database using the DB browser or refresh it using 'Update Medias library DB' via RS WebGUI, same result.

 

SVC-media-Database.jpg

Edited by hk_ben
Link to comment
×
×
  • Create New...