GoDanno
Posts: 27
Joined: Tue Mar 28, 2006 4:34 pm

Postby GoDanno » Thu Apr 23, 2009 6:18 am

I have three channels defined to use the same Channel Plugin. The only difference is the url. During updates one or two of the three channels always fail with the message

Can't determine the feed type.


If I update the failed channels individually, they succeed.

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Thu Apr 23, 2009 7:59 am

GoDanno wrote:I have three channels defined to use the same Channel Plugin.

Is it one of ours or one you've written yourself?

GoDanno wrote:Can't determine the feed type.

This error message means that Awasu can't figure out if it's been given RSS, RDF or Atom, usually because it's not a feed at all, or it's malformed XML.

Check the last feed for the channels that get the error to see exactly what the problem is (<i>Edit|Show last feed</i> menu item, or open the channel's Properties dialog and Shift-click on the <i>Show feed</i> button in the Details tab).

GoDanno wrote:If I update the failed channels individually, they succeed.

Every time Awasu launches an external program, new temp files are created to capture its output (stdout and stderr) so the chance of plugins interfering with each other is minimal. My guess is that the problem is with the plugin itself, may using the same temp file instead of creating a new one for each invocation, or something like that.

GoDanno
Posts: 27
Joined: Tue Mar 28, 2006 4:34 pm

Postby GoDanno » Sat Apr 25, 2009 5:09 am

support wrote:Is it one of ours or one you've written yourself?

It's one I wrote.

support wrote:My guess is that the problem is with the plugin itself.

I'm not sure. I thought I resolved the issue by replacing AltovaXML, which is what I was using to perform an XSLT transfomation, with Saxon. However, I'm still experiencing the error.

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Sat Apr 25, 2009 5:36 am

GoDanno wrote:However, I'm still experiencing the error.

Were there any clues in the "last feed XML" for the erroring channels?

I would try stripping the plugin down so it does nothing except print out a static feed. If the problem goes away, that suggests that the problem is with the plugin or a component it is using. I do a lot of stress-testing in this regard so I'm pretty confident Awasu is working correctly :whistle:

GoDanno
Posts: 27
Joined: Tue Mar 28, 2006 4:34 pm

Postby GoDanno » Sat Apr 25, 2009 9:17 pm

support wrote:Were there any clues in the "last feed XML" for the erroring channels?

I'm afraid there weren't any useful clues. It either showed the correctly formatted xml or the error message only.

:idea: Since my last post, I made one change to the application and that appears to have solved the problem. The channels are updated every 3 hours. I also manually clicked the "Update all channels" button several times during his period as well. Since I made the change, there haven't been any errors.

It doesn't make sense to me but even though Awasu was running three instancees of the console application, the processes weren't necessarily isolated. So, the solution was to use a critical section (the equivalent of the C# lock statement implemented using the .NET System.Threading.Monitor class) in the Main method.

:D I'm a happy camper again!

GoDanno
Posts: 27
Joined: Tue Mar 28, 2006 4:34 pm

Postby GoDanno » Sat Apr 25, 2009 11:40 pm

:( False alarm. It's not resolved.

After rebooting and waiting several hours, the errors reappeared. "Show last feed" reveals the following:

Code: Select all

<head>
<meta>
</head>
<html><body>
<p>The script didn't generate any output.</p>
</body></html>


I will next output a static feed and let you know what happens.

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Sun Apr 26, 2009 9:29 am

GoDanno wrote:The script didn't generate any output.

You get this when the script doesn't print anything out to stdout, and will also generate the "can't determine the feed type" error, since what you're seeing as the last feed is obviously not a valid feed.

This is where it gets interesting... :wall:

One of the things that came out of the aforementioned extensive stress-testing was that Windows would sometimes exhibit some utterly bizarre behaviour where a plugin would generate it's output, it got saved in the temp file Awasu created to capture it, but when Awasu went to read the file, it would be empty, or full of zeroes. This was insanely difficult to track down since it was very intermittent and also very transient; if you waited a second or two after the plugin finished, the file contents would mysteriously become available.

I put something in to work around the problem but it may be the case that this is what's causing your problem. IIRC, this didn't have anything to do with running multiple plugins simultaneously, it would happen even with just one plugin running. The way to check if this is what's happening is to be absolutely sure that the plugin is writing to stdout (e.g. I would build the feed in a string buffer and then print it to stdout and another temp file), then wait for the channel to get the error. If you get the "script didn't generate any output' error but you're sure that it did, then what I described above might be what's happening.

Also, let me know what version of Windows and how many CPU's you have.

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Sun Apr 26, 2009 9:50 am

BTW, this is the code Awasu uses to run external programs.

Code: Select all

SECURITY_ATTRIBUTES securityAttributes ;
memset( &securityAttributes , 0 , sizeof(securityAttributes) ) ;
securityAttributes.nLength = sizeof(securityAttributes) ;
securityAttributes.bInheritHandle = TRUE ;
HANDLE hStdout =
    CreateFile
    (
        pStdoutFilename ,
        GENERIC_WRITE , FILE_SHARE_READ|FILE_SHARE_WRITE ,
        &securityAttributes ,
        CREATE_ALWAYS ,
        FILE_ATTRIBUTE_NORMAL ,
        NULL
    ) ;
if ( hStdout == INVALID_HANDLE_VALUE )
    throw ...

// ...same thing for stderr...

STARTUPINFO startupInfo ;
memset( &startupInfo , 0 , sizeof(startupInfo) ) ;
startupInfo.cb = sizeof(startupInfo) ;
startupInfo.dwFlags = STARTF_USESHOWWINDOW | STARTF_USESTDHANDLES ;
startupInfo.wShowWindow = showWindow ? SW_SHOW : SW_HIDE ;
startupInfo.hStdOutput = hStdout ;
startupInfo.hStdError = hStderr ;
PROCESS_INFORMATION processInfo ;
DWORD rc = CreateProcess( NULL , pCommand ,
                          NULL , NULL , TRUE ,
                          CREATE_NO_WINDOW ,
                          NULL , pExecDir ,
                          &startupInfo ,
                          &processInfo
                        ) ;
if ( ! rc )
    throw ...

// WORKAROUND: closing the handles *before* waiting for the process to finish seemed to make the problem go away...
CloseHandle( hStdout ) ;
CloseHandle( hStderr ) ;

// wait for the plugin to finish
DWORD rc = WaitForMultipleObjects( 1 , &processInfo.hProcess , FALSE , dwTimeout ) ;

It looks like a synchronization problem within Windows: WaitForMultipleObjects() returns when the process finishes, but before Windows gets a chance to flush its buffers and commit the plugin's output to the temp file Awasu created to capture it. The plugin's output does eventually make it to the file but Awasu seems to be going too fast and checking the file before it's ready, sees that the file is empty and thus raises a "script didn't generate any output" error.

But I'm also wondering if there's something wrong with the way I opened the file or created the process. Maybe someone here understands the many subtleties of CreateFile() and CreateProcess() better than I do... :wall:

GoDanno
Posts: 27
Joined: Tue Mar 28, 2006 4:34 pm

Postby GoDanno » Sun Apr 26, 2009 5:27 pm

support wrote:The way to check if this is what's happening is to be absolutely sure that the plugin is writing to stdout (e.g. I would build the feed in a string buffer and then print it to stdout and another temp file), then wait for the channel to get the error. If you get the "script didn't generate any output' error but you're sure that it did, then what I described above might be what's happening.

I'm afraid that this is indeed what is happening. I'm writing the rss feed to the console. Then, within the same block of code, I'm writing the same feed to a temp file. I confirmed that the temp file is there with the correct contents, but I'm still experiencing this intermittent error.

support wrote:Also, let me know what version of Windows and how many CPU's you have.

Windows Vista Ultimate Edition 64-bit
Intel Core 2 Duo T9400 @ 2.53GHz (single CPU w/2 cores)
Awasu 2.4

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Sun Apr 26, 2009 9:47 pm

GoDanno wrote:I'm afraid that this is indeed what is happening. I'm writing the rss feed to the console. Then, within the same block of code, I'm writing the same feed to a temp file. I confirmed that the temp file is there with the correct contents, but I'm still experiencing this intermittent error.

Bugger :mad:

If you get yourself running 2.4.1.alpha1, I'll get a special build out to you that will try to sort this problem out. You have the Pro Edition?

GoDanno
Posts: 27
Joined: Tue Mar 28, 2006 4:34 pm

Postby GoDanno » Sun Apr 26, 2009 10:29 pm

support wrote:If you get yourself running 2.4.1.alpha1, I'll get a special build out to you that will try to sort this problem out. You have the Pro Edition?

I just updated to 2.4.1.alpha1. Yes, I am using the Pro Edition.

MiKylie
Posts: 1
Joined: Fri Oct 30, 2009 8:24 pm
Contact:

Postby MiKylie » Fri Oct 30, 2009 8:32 pm

GoDanno wrote:
support wrote:If you get yourself running 2.4.1.alpha1, I'll get a special build out to you that will try to sort this problem out. You have the Pro Edition?

I just updated to 2.4.1.alpha1. Yes, I am using the Pro Edition.

I have the same problem, but with not Pro Edition. I hope it's possible to fix it easily. TIA.

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Fri Oct 30, 2009 10:14 pm

MiKylie wrote:I have the same problem, but with not Pro Edition. I hope it's possible to fix it easily. TIA.

This was fixed in 2.4.1.


Return to “Awasu - Bug Reports”

Who is online

Users browsing this forum: No registered users and 2 guests