Jump to content

Auto-Timers & Recording History


Griga

Recommended Posts

Currently development is focusing on the DVBViewer Media Server. In the next release there will be substantial changes concerning the database handling. In order to keep you prepared and avoid states of sudden shock :) I decided to post a change log preview and two screen shots illustrating changes concerning the auto-timer handling.

 

Please ask questions (if there are any) or point out aspects that may have not been considered sufficiently.

  • Change: Recording Database: The Recording History database table is not updated and (by default) not used for auto-timer creation anymore. The Media Server now uses the main database table for it, where removed recordings persist (and already persisted in the past) as “disabled” entries until they are deleted manually in the Desktop Web Interface (see below) or by a database cleanup. The needless “Clear / Rebuild Recording History” items have been removed from the tasks page. However, the functions are still available as API URLs (/api/tasks.html?action=ClearRecordedHistory and RebuildRecordedHistory).

  • Added: Desktop Web Interface: The Recordings page provides an additional category “Removed”. It lists removed recordings and allows to edit and selectively delete these items. They represent “disabled” recording database entries that are useful for auto-timer creation (see below).

Zwischenablage01.png

  • Change/Added: EPG Search: By default the “Deactivate auto-timers with already present recording title/subheading” options in the web interface do not refer to the deprecated Recording History database anymore, but to the main database table. An additional “Include removed recordings” checkbox lets the Media Server optionally check auto-timers against removed recordings that are still present as “disabled” entries in the database (see above). If required the additional usage of the recording history database can be reactivated by the “Consider recording history on auto-timer creation” tweak (launch DMSTweaker.bat). It only applies if “Include removed recordings” is ticked.

Zwischenablage02.png

 

Link to comment

Yeah....these sounds to be some solid changes, nicely done Griga :thumbsup:

1. Finally, this could imply I can stop using my db backup system with "dummy.ts" in order to avoid re-run recordings of already removed recordings. The problematic part with these changes in mind, is if it might be a way to rebuild the recordings database with actual as well as removed recordings to keep auto-timer re-run deactivation working for both? The "Update Database" task does both yes? My current recorded history has nearly 4000 entries, which if renders useless forces me to start over from scratch, which would sadden me deeply (havent recorded a re-run in many years);) What im getting at is if there is way to tic a recording as removed/disabled?

2. Does removed webinterface database entries show up if viewing All, Last 30 or By Series? In my case I would feel Im getting a better overview, a recorded history for a series for example,  what has previously been recorded no matter of it still remains or is removed. Just having a disabled status indicator on it would be nice so you clearly can tell a part a recording that has been removed or is alive. Will removed recordings just be located/passed to "Removed Recordings"? If the webinterface recordings database sometime in the future will be sortable (a sorely missed functionality) im sure I would love to sort all series recordings regardless of being removed or not. Perhaps this could be a filtering option "show removed recordings everywhere" or "Keep them in Removed Recordings only", but that would imply messing around with the javascript which nobody except maybe janee could pull off. Hence Im using my own software to make up for the shortcomings of the DMS Recording tab.  

3. Are the recordings thumbnail removed at the webinterface recording entry tooltip when it has "disabled" status or does the thumbnail gets removed upon manually entry deletion/clean up database task?  

 

Edited by majstang
Link to comment
11 hours ago, majstang said:

if it might be a way to rebuild the recordings database with actual as well as removed recordings to keep auto-timer re-run deactivation working for both?

 

There is no way to rebuild the database from the scratch including  removed recordings. If the recording and the associated txt file and the corresponding database entry are gone, all information about the recording is gone and cannot be reconstructed.

 

However, DB entries for removed recordings can exist without the corresponding files as long as they are not deleted manually and no DB cleanup is performed. A DB update does not remove them. This is not new, but was always (invisibly) handled in this way. If you didn't perform a DB clean up for some time, there may be a lot of removed recordings in your DB main table, but with the enabled flag set to 0, so they don't show up in the UI.

 

If you want to have a look at it right now open the SvcDatabase.db3 with the SQLite Browser. The warning that pops up on start can be ignored unless you use the browser for changing the database. Select the "Browse Data" tab and the table that you want to inspect. "recordings" is the main table and "recordedlist" the recording history.

 

11 hours ago, majstang said:

My current recorded history has nearly 4000 entries, which if renders useless forces me to start over from scratch, which would sadden me deeply

 

The recording history will not be updated anymore by future DMS versions, but not be deleted. You may use the tweak mentioned in my first post to let it still take effect in addition to the new mechanisms.

 

Unfortunately it is not possible to reconstruct main table entries from the history table because it is very limited (the SQLite Browser will make it obvious). With other words: It is not possible to transfer history entries back to the main recordings table and make them visible. Too many data is missing.

 

11 hours ago, majstang said:

Does removed webinterface database entries show up if viewing All, Last 30 or By Series?

 

No. Only in the "Removed" category (current state). However, a "mixed" view could be considered. From a technical point of view that's  no problem. Rather the manageability.

 

11 hours ago, majstang said:

 If the webinterface recordings database sometime in the future will be sortable

 

It is already now by categorisation, but not in the usual way by clicking column headers like on the Timers page. Dunno if this can be accomplished somehow and if it complies with the category selection. I will have a look at it later (maybe next but one release).

 

11 hours ago, majstang said:

Are the recordings thumbnail removed at the webinterface recording entry tooltip when it has "disabled" status

 

Not necessarily. Thumbnails are deleted on a database cleanup (that removes entries with "disabled" status anyway) and when a recording is deleted via web interface or API. But deleting a recording on disk and performing a database update preserves the thumbnail (I've just tried...).

 

Deleting an existing recording via web interface moves the database entry to the "Removed" category. Deleting entries in the "Removed"  category lets them vanish forever (thus allowing to perform a selective cleanup).

 

BTW: The possibility to edit recordings in the "Removed" section will allow for some nice dirty tricks. E.g. you may recreate a lost txt file from the database entry by ticking the "Update Info File" checkbox and clicking Save. By changing the title and subheading of some removed recording you can create artificial "timer disablers" without ever having them recorded. And things like that...

 

Link to comment
On 2017-11-21 at 7:54 AM, Griga said:

There is no way to rebuild the database from the scratch including  removed recordings. If the recording and the associated txt file and the corresponding database entry are gone, all information about the recording is gone and cannot be reconstructed.

 

However, DB entries for removed recordings can exist without the corresponding files as long as they are not deleted manually and no DB cleanup is performed. A DB update does not remove them. This is not new, but was always (invisibly) handled in this way. If you didn't perform a DB clean up for some time, there may be a lot of removed recordings in your DB main table, but with the enabled flag set to 0, so they don't show up in the UI.

That is perfect:) My db backup files (dummy.ts (including the original recording fileinfo) .txt and .log) contains the sufficient stuff to rebuild the database form scratch, although when that is done DMS will treat them as actual recordings. Then I will have to change the enabled flag to 0 via the SQLLite editor to get them the disabled status. Hopefully manually messing with the database like that would not corrupt it (had such experiences in the past with RS). 

 

On 2017-11-21 at 7:54 AM, Griga said:

No. Only in the "Removed" category (current state). However, a "mixed" view could be considered. From a technical point of view that's  no problem. Rather the manageability.

Yeah, that would be great, cuz after a while (several 1000 recordings done) your database will mainly consist of removed recordings and not having them DMS style sortable would be unfortunate. They are IMO quite important still as long as they prevents recordings of re-runs. 

 

On 2017-11-21 at 7:54 AM, Griga said:
On 2017-11-20 at 8:23 PM, majstang said:

 If the webinterface recordings database sometime in the future will be sortable

 

It is already now by categorisation, but not in the usual way by clicking column headers like on the Timers page. Dunno if this can be accomplished somehow and if it complies with the category selection. I will have a look at it later (maybe next but one release).

Sorry for going slightly off-topic with this, but it is related to the recordings database at least. Yeah, sorting season/episode numbering by clicking the "Name" column header might work. Im talking about a proper sorting of the recordings database "Info" column which for a series database is HIGHLY indispensable. I know Delphi (the XEPG programmer) has these numberings in the title and I use them in the subheader. Airing of series episodes are often done in non-chronological order (some broadcasters do it like that) and having the opportunity to sort the S01E01 would be great. Here is a sample from my own software with time-reverse base sorting and filtering the Mythbuster show with season/episode numbering sort:

 

Skärmklipp.PNG

Edited by majstang
Changed considerations
Link to comment
28 minutes ago, majstang said:

although when that is done DMS will treat them as actual recordings. Then I will have to change the enabled flag to 0 via the SQLLite editor to get them the disabled status.

 

You don't have to. Just let the DMS read your fake stuff as recordings, then remove it and perform another database update. This will set enabled = 0 for all corresponding entries.

Link to comment

About series recordings

 

If you want to record series having episode numbers in the title and only check "Recording Title" in "Deactivate auto-timers with the same" it can result in too many recordings. It is described here:

 

http://www.DVBViewer.tv/forum/topic/54879-recording-service-search-epg/?tab=comments#comment-410355

 


I have just tested with DMS 2.0.4, the problem persists.

 

If you additionally check against "Timer Name" you get the expected result. That is OK if you know, but not so user friendly.

 

There is a bit more to it

 

At least for external EPG, the Recording Event (used to construct the timer name) is often some random text and then not good for deciding whether a timer should be disabled or not in case of a series. If Xepg is used to put the episode number at the end of the the title or the native EPG has episode numbers in the title a safe way to record series should be possible.

 

To illustrate the situation as is now:

 

huset_search.jpg.3291f475f19a3082ebda375c78ed641c.jpg

 

Just to quickly get an example I have changed the event "Mellem himmel og jord" to "Mellem" for one programme item in the xmltv file. The Create Auto Timers for the upper example give

 

huset_timers.jpg.086aa96be7f7fce065aeb097cee686b3.jpg

 

This is not what is wanted in case of a series recording. Such a problem would not exist if the first mentioned issue in this post was solved.

 

Link to comment

I can see the problem. You want already created timers to provide the associated EPG title as property so it can be checked against the title of further EPG search results, or with other words, you want timers to be handled as if the recording was already done, right?

 

I'd call it betting on the future. What if executing the first timer fails for some reason? You would be happy about a backup timer that records a repetition of the programme, wouldn't you?

 

Link to comment
1 hour ago, Griga said:

or with other words, you want timers to be handled as if the recording was already done, right?

 

No, I want it handled for what it is: If a timer allready exists for an episode of a series then another timer for the same episode should not be created.

 

1 hour ago, Griga said:

I'd call it betting on the future

 

If "Timer Name" is checked it works that way. Would be betting on the future as well.

 

1 hour ago, Griga said:

You would be happy about a backup timer that records a repetition of the programme, wouldn't you?

 

No. The extra timers are executed even if no recordings fail (the usual case).

Link to comment
3 minutes ago, Delphi said:
 
1 hour ago, Griga said:

or with other words, you want timers to be handled as if the recording was already done, right?

 

No, I want it handled for what it is: If a timer allready exists for an episode of a series then another timer for the same episode should not be created.

 

A timer is an assumed future recording, isn't it? But you never know for sure...

 

5 minutes ago, Delphi said:

If "Timer Name" is checked it works that way. Would be betting on the future as well.

 

OK, should be dropped. Doesn't seem to be useful anyway.

 

7 minutes ago, Delphi said:

No. The extra timers are executed even if no recordings fail (the usual case).

 

Your backups are executed even if your hard disk doesn't crash (the usual case).

Windows updates are executed even if no malware tries to infect your PC  (the usual case).

Link to comment
41 minutes ago, Griga said:
1 hour ago, Delphi said:

If "Timer Name" is checked it works that way. Would be betting on the future as well.

 

OK, should be dropped. Doesn't seem to be useful anyway.

It might be a road to trouble, I think Lars introduced check against "Timer Name" cuz neither of check against Rec.title or Rec.event was abled to deactivate existing timers, only new ones created via the Autotimer. I may remember it wrong though.

Link to comment
17 minutes ago, majstang said:

I think Lars introduced check against "Timer Name" cuz neither of check against Rec.title or Rec.event was abled to deactivate existing timers

 

Is the timer name always a combination of title and even? I'll check it tomorrow...

 

11 minutes ago, Delphi said:

IMO it's not relevant to compare creating recording timers to scheduling backups/windows updates the way you are trying to.

 

If a recording succeeds or not may make or break a happy family life. I know this from several posts in the German forum saying "I don't really mind, but my wife, you know..." A missing series recording may even kill a marriage. A missing Windows update usually won't. So you are right. It cannot be compared.

 

Anyway, I feel that the title / event (that I'd rather call subheading) comparison should apply to recordings and timers. The main purpose of this function is to avoid unwanted recordings, and the expectations of average users (like Delphi :)) should be met. It's hard for them to understand why timers are not included. The timer name comparison should be dropped, but there should be a checkbox "Include timers" besides "Include removed recordings", so that everybody who wants (particularly backup fanatics) can limit the scope of the title/subheading or whatever comparison.

 

This will break compatibility with previous versions, though.

 

 

Link to comment
7 hours ago, Griga said:

Anyway, I feel that the title / event (that I'd rather call subheading) comparison should apply to recordings and timers.

 

Regarding the preconditions for such a change:

  • By default the timer name is composed of [title] - [subheading], if (and only if) the timer is created via EPG. The name is persistent (stored in the svctimers.xml) and remains unchanged, unless it is edited by the user.
  • In contrast to the timer name EPG data is assigned dynamically on demand to a timer, which means, it is not persistent (not stored in the svctimers.xml) and not guaranteed to remain the same over time. The DMS uses different criteria to find the matching EPG data (PDC if present, otherwise a combination of start/endtime and Event ID comparison). That's why a comparison with a timer's current EPG data is not fully reliable and may yield unexpected results.
  • The initial EPG title/subheading that is used by the DMS for timer creation (automatically or manually) could be stored temporarily during runtime in the timer object for comparison purpose. This would restrict it to timers that are created within the same DMS session, however. After a DMS stop / start the data would be gone.
  • The initial EPG title/subheading stored in timer objects could be stored in the svctimers.xml to make it persistent.
  • Even if the timer's initial EPG title/subheading are stored permanently timers created via /api/timeradd.html are still excluded from comparison  (particularly concerning DVBViewer) unless the API is extended and clients / add-ons adapt to it.

That's a good example for the dependencies / obstacles showing up on conceptual design in the DMS/DVBViewer environment. ;)

 

Link to comment
8 hours ago, Griga said:
8 hours ago, majstang said:

I think Lars introduced check against "Timer Name" cuz neither of check against Rec.title or Rec.event was abled to deactivate existing timers

 

Is the timer name always a combination of title and even? I'll check it tomorrow...

Yes, "Timer name" function doesnt seem make any distinction between title and event and I think that is the root to Delphis problem. The Rec.title function wants to deactivate the second "Mellem" timer but gets overridden by the "Timer name" function. Just my guesses and visions of it;)

Link to comment
6 minutes ago, majstang said:

The Rec.title function wants to deactivate the second "Mellem" timer but gets overridden by the "Timer name" function.

 

Wrong. The Rec.title function doesn't take effect at all because the recording is not yet done. That's why Delphi wants timers (= future recordings) to be handled like already executed recordings, and that's exactly what I considered above.

 

Link to comment
15 hours ago, Griga said:

the expectations of average users (like Delphi :)) should be met. It's hard for them to understand why timers are not included.

 

For average programmers (like Griga :)) it's hard to understand when a code/datastructure change is needed.

Link to comment

Relax and take it with a smile(y).

 

On 23.11.2017 at 12:12 AM, Griga said:

Anyway, I feel that the title / event (that I'd rather call subheading) comparison should apply to recordings and timers.

 

I think it should look like this in the web interface:

 

Zwischenablage02.png

 

The main purpose is to make these options comprehensible / usable without expert knowledge, not to spare all you lazy guys a few mouse clicks for activating/deactivating auto-timers manually or deleting surplus recordings on your TB hard disk.

 

A "cheap" solution for checking against the EPG title / subheading of already existing timers is to (re-)use the timer name, because by default it is composed of the EPG title and subheading separated by " - ", where the subheading is shortened to 50 characters if it is longer. The EPG title can be checked against the beginning of the timer name (plus check for same length or following separator) and the EPG subheading against the end of the timer name (plus check for preceding separator).

 

This is not a perfect solution, but probably sufficient in practice. It is half-compatible with DVBViewer since by default it uses the EPG title (without subheading) as timer name.

 

A by-product is the possibility to manually create timers with a certain name and a start date far in the future that only serve as auto-timer disablers.

 

Open questions:

  • Shall the comparison be case sensitive? It wasn't up to now for recordings, but for timers. I'd say no.
  • Shall disabled timers be included in the check? They were up to now. I'd rather say no.
  • Shall "Tune Only" timers be included? They were up to now. I'd say no.
  • Shall auto-repeating timers be included? They were up to now.

Something else concerning these options that must be considered?

 

Edited by Griga
Link to comment
17 hours ago, Griga said:

The main purpose is to make these options comprehensible / usable without expert knowledge, not to spare all you lazy guys a few mouse clicks for activating/deactivating auto-timers manually or deleting surplus recordings on your TB hard disk.

I think this is exellent, much more clear and descriptive than ever. Well, making the timer deactivation system perfect is impossible, too many unforeseen factors that could break it, which implies if you wanna use it still requires monitoring and manually activating/deactivating of recording timers after the Auto-timer done its magic. My problems relates to which external EPG source im forced to use (there are not many swedish ones using season/epsiode numberings with episode names for series) and if they do they are not consistent. That often results in series episodes, where season/epsiode numberings with episode names suddenly goes missing, resulting in recording timers (with only title present) gets automatically deactivated cuz such a show has been recorded before (which I obviously want). There is nothing I can do about it atm (or anyone else for that matter) except see to it some monitoring and manual labour are done after the weekly auto-timer, to avoid domestic outbursts/wars that would be the case when a wanted show gets auto-deactivated and never was recorded;) The great part is its much less of the manual labour compared to how it was before this system was introduced:)

 

17 hours ago, Griga said:

A by-product is the possibility to manually create timers with a certain name and a start date far in the future that only serve as auto-timer disablers.

Yes, that is a truly great by-product which im using extensively with perfect results. 

 

17 hours ago, Griga said:

Shall the comparison be case sensitive? It wasn't up to now for recordings, but for timers. I'd say no.

It would be preferrable if you have a recording named "Gold rush Alaska - S01E01" the recording timer "Gold Rush Alaska - S01E01" still gets deactivated. This is a persistent issue with inconsistent external EPG sources. 

 

17 hours ago, Griga said:

Shall disabled timers be included in the check? They were up to now. I'd rather say no.

Can you elaborate this case? I was not aware of disabled timers could be auto-activated. Maybe that was a feature Lars intended to develop (lehtikissa suggests it herehttp://www.DVBViewer.tv/forum/topic/60827-auto-timers-recording-history/ Although it may be very hard to code.

 

17 hours ago, Griga said:

Shall "Tune Only" timers be included? They were up to now. I'd say no.

I agree, I dont see any point with including "Tune Only" timers.  

 

17 hours ago, Griga said:

Shall auto-repeating timers be included? They were up to now.

You mean internal and process task timers was included? That was a surprise and its hard to see the point with that too. 

 

17 hours ago, Griga said:

Something else concerning these options that must be considered?

Nothing that comes to mind.

Edited by majstang
Link to comment
16 hours ago, Griga said:
  • Shall the comparison be case sensitive? It wasn't up to now for recordings, but for timers. I'd say no.
  • Shall disabled timers be included in the check? They were up to now. I'd rather say no.
  • Shall "Tune Only" timers be included? They were up to now. I'd say no.
  • Shall auto-repeating timers be included? They were up to now.
  • No
  • No
  • No
  • No
Link to comment
3 hours ago, majstang said:

Can you elaborate this case?

 

New auto-timers (as result of an EPG search) are checked against the list of already existing timers in order to decide whether they must be disabled or not. Only recordings timers from the list are used for the EPG title / subheading comparison. Other timer types (e.g. task timers) are ignored.

 

However, already existing recording timers in this list may be disabled, either manually by the user in the web interface or by a previous auto-timer creation session (which means, they stay in the list until their due time, but are not executed). The question is if already existing disabled recording timers in the list should be be used for EPG title / subheading comparison.

 

Auto-repeating recording (!) timers are not deleted after execution, but the start time is shifted to the next scheduled day. If you don't delete them manually they will trigger recordings forever on a daily/weekly basis. See Desktop Web Interface -> Timer Edit Window -> day of the week checkboxes. The recorded content may vary, but the timer name remains the same. They can be used for programmes that are broadcasted daily or on certain days of the week at the same time. A single auto-repeating timer covers them all. The question is if auto-repeating recording timers in the list should be be used for EPG title / subheading comparison.

 

I hope this clarifies things a bit...

Link to comment
1 hour ago, Griga said:

Auto-repeating recording (!) timers are not deleted after execution, but the start time is shifted to the next scheduled day. If you don't delete them manually they will trigger recordings forever on a daily/weekly basis. See Desktop Web Interface -> Timer Edit Window -> day of the week checkboxes. The recorded content may vary, but the timer name remains the same. They can be used for programmes that are broadcasted daily or on certain days of the week at the same time. A single auto-repeating timer covers them all. The question is if auto-repeating recording timers in the list should be be used for EPG title / subheading comparison.

Aha, this is the old way of doing series recordings (timer has "Timer enabled Web" status) and no there isnt any point with doing the EPG title / subheading comparison for those. Allowing that might do more damage than good for this recording option. 

 

1 hour ago, Griga said:

However, already existing recording timers in this list may be disabled, either manually by the user in the web interface or by a previous auto-timer creation session (which means, they stay in the list until their due time, but are not executed). The question is if already existing disabled recording timers in the list should be be used for EPG title / subheading comparison.

Allowing EPG title / subheading comparison for these are dependent on whether there are plans to develop an auto-reactivator function (lehtikissa) yes, if there arent such plans no. 

Link to comment

What if an ongoing recording is canceled and deleted? Shall the recording database remember it as removed entry?

 

Current state:

 

No if an ongoing DMS recording is deleted by a DVBViewer client (that delegates timers to the DMS).

Yes if an ongoing DMS recording is deleted via web interface.

 

I'd rather say no.

 

Link to comment
12 hours ago, Griga said:

What if an ongoing recording is canceled and deleted? Shall the recording database remember it as removed entry?

 

I wanna say yes on this one, cuz it might be a source of confusion having these kind of exceptions to the normal and expected behaviour. Although its a neat idea not have to manually remove it from removed, cuz it is easy to forget doing that and if you dont do it a future recording timer might be deactivated in vain. The "no" option might be rather tricky to get right. First checking if the actual recording does match the timer specified start and end time (if it was cancelled prematurely) and then I wonder if timings between cancellation and deletion might be a factor to take into account. Probably I do complicate things and it is much easier to accomplish once knowing the source code.

Link to comment
13 hours ago, Griga said:

What if an ongoing recording is canceled and deleted? Shall the recording database remember it as removed entry?

 

It'd be more logical not to remember it as a removed entry as an ongoing recording that is removed before it is finished in a sense is not yet a recording.

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...