XEPG does not create new timers

[x112]

Since one of the last DMS Updates XEPG does not create new timers anymore. If i call the task "Create autotimers" in the web gui manually then the timers are created.

 

My workflow is:

Process Timer in DMS calls EPGbuddy

EPGBuddy loads the XML Data in one XML File and calls XEPG

XEPG clears external EPG data, imports the XML File and calls Autotimer after import

 

But there are no new timers created until i call it manually again.

In the past this worked without problems.

[Griga]

http://localhost:8089/api/tasks.html?task=AutoTimer works in DMS 3.1, just tested...

 

[Delphi]

Works when called from Xepg here as well using DMS 3.1, just tested...

[x112]

vor 50 Minuten schrieb Griga:

http://localhost:8089/api/tasks.html?task=AutoTimer works in DMS 3.1, just tested...

 

This is what i meant with "Create autotimers" in the web gui manually.

But why are there no new timers after the process timer?

Here are my XEGP Config and Logfile.

files.zip

[Griga]

vor 1 Minute schrieb x112:

This is what i meant with "Create autotimers" in the web gui manually.

 

I've posted an API URL intended to be used by add-ons. It is not used by the DMS web interface. So what I've tested is if add-ons can trigger auto timer creation in DMS 3.1. Obviously they can.

 

[x112]

I think it has something to do with the "clear EPG" option in XEPG. I also get timers based on the DVB EPG and not on the external EPG.

 

 

If i delete this timer and call the auto timer task then i get this timer (disabled is correct).

 

[x112]

It looks like this is a timing problem. If i do not use "clear EPG" in XEPG, the timers are created as expected. The EPG XML file is 42MB. Could it be that XEPG calls the Autotimer Task before DMS has processed the imported EPG?

[Delphi]

You are right! Well observed

 

The problem is that the DMS needs some time to digest the imported EPG. I have attached a new version of Xepg which allows to set a delay before the autotimer is called. The Xepg.zip only contains Xepg.exe, just replace that in your current installation.

 

The default value is 20 seconds. Works fine  here with 10 seconds.

 

Xepg.zip

Edited May 7, 2021 by Delphi

[Griga]

vor einer Stunde schrieb Delphi:

The problem is that the DMS needs some time to digest the imported EPG.

 

The DMS processes the imported data in a separate thread, so sending the data may be finished from the client's point of view before processing it is finished.

 

Handling it with an (estimated) delay is a gamble. I think we should introduce a new API call that returns the state of the import process (busy / finished), that Xepg can query every 10 seconds or so.

 

[x112]

Thanks for the quick fix. In my case 40s are necessary (it does not work with 30s). I have a new system with a Intel Core i5 11500 cpu, but 47429 "Paired <programme> items" need some time to process.

 

I am using "clear EPG" because Nickelodeon and Super RTL have a very "dynamic" EPG. In most timers i am using "today"-"Today+1" as the search date range.

 

[Delphi]

1 hour ago, Griga said:

I think we should introduce a new API call that returns the state of the import process (busy / finished), that Xepg can query every 10 seconds or so.

 

A great Idea. I can use this for another delay "Pause before Xepg Standby/Hibernate" as well.

Do you think I should have a configurable timeout value? (maybe 200 seconds by default)

[Delphi]

1 hour ago, x112 said:

 

I am using "clear EPG" because Nickelodeon and Super RTL have a very "dynamic" EPG. In most timers i am using "today"-"Today+1" as the search date range.

 

So do I. The best settings IMO.

[Griga]

vor 9 Stunden schrieb Delphi:

A great Idea. I can use this for another delay "Pause before Xepg Standby/Hibernate" as well.

 

Alternatives:

• The DMS could as well delay the execution of auto-timer requests (even of those triggered manually) until an import is finished. Add-Ons wouldn't have to take care of it. However, they would never come to know when the job is done.
• Xepg could append a unique "magic" EPG item (e.g. with a special (random?) EPGChannelID and EventID) to the imported data and try to read it back by using /api/epg.html until it is available. This would work with all DMS versions, but would not be so efficient and easy to handle as a special API call.

vor 10 Stunden schrieb Delphi:

Do you think I should have a configurable timeout value? (maybe 200 seconds by default)

 

There guessing starts again... how long may it take? It's not just the execution time. The EPG import thread additionally pauses for 1 millisecond (Sleep(1)) after having processed 10 items, particularly to keep single core PCs responsive. Sleep lets other threads kick in and do their job.

 

Let's assume a timer with a 10 seconds interval that triggers repeated queries for the import state. The timerwill be stopped / disabled in case of success (import finished) or if the DMS is not reachable (HTTP connection timeout). I cannot imagine conditions under which the import thread would never come to an end, e.g. due to an endless loop. However, it may be wise to also expect the unexpected. Though it wouldn't be harmful if the timer continues until Xepg gets closed, a timeout of let's say 10 minutes may be appropriate.

 

[Delphi]

On 5/7/2021 at 7:58 PM, Griga said:

I think we should introduce a new API call that returns the state of the import process (busy / finished), that Xepg can query every 10 seconds or so.

 

I prefer that. It's more clean and more flexible than your alternatives. If a HTTP error happens I will not call the AutoTimer. Same if I decide to use a TimeOut value and a timeout happens (10 minutes is a good default value IMO). The timeout value can depend on the context (Standby/Hibernate or AutoTimer) and could be configurable by the user. 

 

The main purpose of Xepg is to import EPG, it's not a catastrophe if the AutoTimer call fails. It will be logged in invisible mode and displayed in windowed mode.

[Griga]

vor 9 Stunden schrieb Delphi:

I prefer that. It's more clean and more flexible than your alternatives.

 

The new API can be implemented in two flavours:

• /api/epgimportstatus.html delivers a simple text that only contains a single integer, indicating the number of running EPG import threads (0 = no import going on, > 0 = busy).
• /api/epgstatus.html delivers a XML with additional EPG information, e.g. the current number of EPG entries (could be used for displaying the progress), the state of the automatic EPG update (active/inactive, next date/time) or something else that you need/want.

 

[x112]

Just a question in this EPG context: is it possible to clear only the external EPG of one channel? I use the clear function only because two channels rolls every day the dice what to write in the EPG.

Edited May 9, 2021 by x112
external inserted

[YARD2]

Bit off Topic ... but would it be possible to write the EPG.dat after import of XEPG ?

 

Edited May 9, 2021 by YARD2

[Delphi]

11 hours ago, Griga said:

/api/epgstatus.html delivers a XML with additional EPG information

I think this is the best, since it can be extended later on if need be (as long as no existing xml node names are changed). For now I will only need the number of running EPG import threads.

Edited May 9, 2021 by Delphi

[Griga]

vor 14 Stunden schrieb Delphi:

I think this is the best, since it can be extended later

 

It will look like this:

 

<epgstatus>
  <epglang>deu</epglang>
  <timezone>120</timezone>
  <importstate>0</importstate>
  <epgcount>184467</epgcount>
  <nextupdate>20210510085523</nextupdate>
  <updatetotalcount>0</updatetotalcount>
  <updatecount>0</updatecount>
</epgstatus>

 

• epglang is the preferred EPG language selected by the user.
• timezone is the current time zone offset in minutes
• importstate indicates the number of active import threads (0 = no import going on).
• epgcount is the overall number of stored EPG entries.
• nextupdate is the date/time of the next automatic EPG update in yyyymmddhhnnss format. The element is empty if no update is scheduled.
• updatetotalcount and updatecount refer to the overall / already scanned number of transponders. 0 means no EPG update is going on.

Anything missing?

 

[Delphi]

Looks good to me 

 

 

Spoiler

Code has changed

 

Edited June 1, 2021 by Delphi
code deleted since it has changed

[Delphi]

A new version of Xepg taking advantage of api/epgstatus.html is available:

https://www.DVBViewer.tv/forum/topic/59663-xepg/?do=findComment&comment=495936

[x112]

The new XEPG version is working: minimum delay seems to be 1 Minute

 

with Delete EPG option:
2021-06-02 08:57:10.076 Waiting until Media Server has processed imported EPG...
2021-06-02 08:57:10.076 Running DMS import threads: 47
2021-06-02 08:58:12.294 Running DMS import threads: 0
2021-06-02 08:58:12.560 DVBViewer Media Server Auto Timer called

 

without Delete EPG option
2021-06-02 09:27:57.676 Waiting until Media Server has processed imported EPG...
2021-06-02 09:27:57.676 Running DMS import threads: 47
2021-06-02 09:28:59.864 Running DMS import threads: 0
2021-06-02 09:29:00.113 DVBViewer Media Server Auto Timer called
 

Thanks to all.