rsr
Posts: 27
Joined: Wed May 06, 2009 10:03 pm

Postby rsr » Sat Aug 15, 2009 3:12 am

I have created some search channels that searches other channels with specific tags. The search channels uses the "Search in: Descriptions" option to search for the channels with matching tags. That way we just need to look in the search channels for feed items that we are interested in. In short, just a way we can quickly group related information.

That part works fine, but there seems to be a timestamp issue that occurs from doing this. The feeds summary displayed in the Item Pane shows the received date/time only and not the published date/time.

If I recover the feed from its original channel, the feed item generates the published date/time value.

I would prefer to obtain the published date/time information instead of the received time. Do you have any ideas why this might happening , or what I may be doing wrong?

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Sat Aug 15, 2009 3:58 am

To understand why this is happening, and thus the solution, you need to understand how search channels work. The solution is at the bottom of this post, if you don't care :-)

The only difference between normal channels that update from a web site and search channels (and also plugin channels) is where the feed items come from. For normal channels, they are simply downloaded from the web site, for search channels Awasu runs the search query and automatically generates a feed item for each search result, for plugin channels Awasu runs the plugin script which itself generates feed items however it wants (e.g. based on what's in a database, Google search results, etc.).

After this happens and the feed items move on through the rest of Awasu, <i>Awasu has no record of where they came from</i>, it's all just a stream of feed items. So for feed items in your search channel, Awasu doesn't know that the feed items originally came from another channel, they're just items that appeared in the search channel's incoming item stream. Hence their timestamp is for when the feed item was received in the search channel, not the timestamp for the original feed item (since there's no record of the relation between the two).

So, that's the "why". Here's the solution:

When a search channel updates, Awasu runs the search query and generates a feed item for each search result. This is template-based and you can find the template in <tt>$/Resources/Search Channel Templates/DefaultSearchChannelFeed.template</tt>. The relevant bit of this file:

Code: Select all

{%REPEAT% SearchResults}
<entry>
    <id>{%ITEM-METADATA% awasuId}</id>
    <link href="{%ITEM-METADATA% url encode=sgml chars="<&\""}" />
    <title>{%ITEM-METADATA% name!}</title>
    <content>{%ITEM-METADATA% description}</content>
    <source>
        <link href="{%OPEN-ITEM-CS-URL% encode=sgml chars="<&\""}" />
        <title>{%CHANNEL-METADATA% name}</title>
    </source>
</entry>
{%/REPEAT%}

For every search result, a new Atom entry is generated, using <tt>{%ITEM-METADATA%}</tt> to insert the appropriate values from the search result feed item.

So, what we need to do is extract the timestamp from each search result feed item and insert it into the generated feed. Awasu will then use that instead of the time received since it's a better quality timestamp. This can be done like this:

Code: Select all

<published> {%ITEM-METADATA% published format="%Y-%m-%dT%H:%M:%SZ" timeZone=0} </published>

Setting <tt>timeZone</tt> to 0 will return the time in UTC, hence the "Z" in the timestamp (Z = Zulu time = UTC).

Atom also has the <tt><updated></tt> tag which specifies when a feed item was last updated (not published), so you would need to do the same thing for that.

So, the final solution is to have something like this:

Code: Select all

{%REPEAT% SearchResults}
<entry>
    <id>{%ITEM-METADATA% awasuId}</id>
    <link href="{%ITEM-METADATA% url encode=sgml chars="<&\""}" />
    <title>{%ITEM-METADATA% name!}</title>
    <content>{%ITEM-METADATA% description}</content>

<published> {%ITEM-METADATA% published format="%Y-%m-%dT%H:%M:%SZ" timeZone=0} </published>
<updated> {%ITEM-METADATA% modified format="%Y-%m-%dT%H:%M:%SZ" timeZone=0} </updated>

    <source>
        <link href="{%OPEN-ITEM-CS-URL% encode=sgml chars="<&\""}" />
        <title>{%CHANNEL-METADATA% name}</title>
    </source>
</entry>
{%/REPEAT%}

If you modify the <tt>DefaultSearchChannelFeed.template</tt> file directly, it'll get overwritten the next time you install a new version of Awasu so you'll have to remember to re-apply the changes. Alternatively, you can change a copy of the file and configure search channels to use that instead by setting the <tt>FeedTemplateFilename</tt> parameter in their <tt>.CHANNEL</tt> file. There's no way to set this across the board but I'll add something for the next release.

I'll probably incorporate this change, or something similar, for the next release. Just need to have a ponder over the larger ramifications of doing so...
Last edited by support on Sun Aug 16, 2009 2:33 pm, edited 1 time in total.

rsr
Posts: 27
Joined: Wed May 06, 2009 10:03 pm

Postby rsr » Sat Aug 15, 2009 4:09 pm

Thanks again for your outstanding support!!

I applied the changes that you recommended, but unfortunately the results are the same. Do you have any other suggestions?

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Sun Aug 16, 2009 12:22 am

rsr wrote:I applied the changes that you recommended, but unfortunately the results are the same. Do you have any other suggestions?

It will only only apply to new items returned in the search channel feed. Deleting and re-creating the search channel will work.

rsr
Posts: 27
Joined: Wed May 06, 2009 10:03 pm

Postby rsr » Sun Aug 16, 2009 12:00 pm

I may still be missing something :(

Modified the template file, deleted all my existing channels and started from scratch. Unfortunately the results are still the same.

My version of Awasu is 2.4.1.alpha3.

The following shows the complete text in the modified DefaultSearchChannelFeed.template file:

<xml>

<feed>

<title>{%CHANNEL-METADATA% name}</title>
<subtitle>{%CHANNEL-METADATA% description}</subtitle>


{%REPEAT% SearchResults}
<entry>
<id>{%ITEM-METADATA% awasuId}</id>
<link href="{%ITEM-METADATA% url encode=sgml chars="<&\""}" />
<title>{%ITEM-METADATA% name!}</title>
<content>{%ITEM-METADATA% description}</content>


<published> {%ITEM-METADATA% published format="%Y-%m-%dT%H:%MZ" timeZone=0} </published>
<updated>{%ITEM-METADATA% modified format="%Y-%m-%dT%H:%MZ" timeZone=0} </updated>


<source>
<link href="{%OPEN-ITEM-CS-URL% encode=sgml chars="<&\""}" />
<title>{%CHANNEL-METADATA% name}</title>
</source>
</entry>
{%/REPEAT%}

</feed>

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Sun Aug 16, 2009 12:25 pm

Hmm, there is one thing I didn't consider that might be causing this. The new template file gets the "published" and "updated" times for each feed item found as a search result and inserts them into the feed items that are being generated for the search channel. This will work, <i>assuming that the search result feed items actually have "published" and "updated" timestamps</i> :wall: If they don't, the generated feed items for the search channel won't have any published/updated timestamps either and so Awasu will continue to show when they were received.

In the Details page of the Properties dialog for the search channel, Shift-Click on the <i>Show feed</i> button to see the raw feed XML. Find the <tt><published></tt> and <tt><updated></tt> tags and check if there's anything inside them.

Also, post (or email) the feed URL of the channel you're expecting to get search results from and I'll give it a go myself.

rsr
Posts: 27
Joined: Wed May 06, 2009 10:03 pm

Postby rsr » Sun Aug 16, 2009 1:08 pm

Yes the raw feed xml shows the published date/time of the feeds

Example:

<published>2009-08-11T23:43Z </published>
<updated> </updated>

My test feed URL is: http://www.embedded.com/rss/rss-all.jhtml

User avatar
support
Site Admin
Posts: 3021
Joined: Fri Feb 07, 2003 12:48 pm
Location: Melbourne, Australia
Contact:

Postby support » Sun Aug 16, 2009 2:29 pm

My misteak :oops: Timestamps must have seconds in them :sorry:

Change the <tt>H:%MZ</tt> to be <tt>H:%M:%SZ</tt>

rsr
Posts: 27
Joined: Wed May 06, 2009 10:03 pm

Postby rsr » Sun Aug 16, 2009 9:38 pm

That worked!!! :lol:

Thankyou!


Return to “Awasu - General Discussion”

Who is online

Users browsing this forum: No registered users and 4 guests