View unanswered posts | View active topics

Reply to topic  [ 17 posts ]  Go to page 1, 2  Next
User avatar

Joined: Mon Sep 08, 2008 3:16 pm
Posts: 226
Location: Elk Grove, California
Post HTTP APIs
A couple of months ago I sent several emails discussing possible HTTP-based APIs with Taka, some of which he said had possibilities, some of which he said he no chance of getting implemented (you can't blame me for trying). But out of all of my suggestions, two of the features requests that I think hold the most benefit to me and others are the 1) ability to run Channel Reports and 2) the ability to issue a search in your feed items.

Why an HTTP API?

1) It's well supported in a wide variety of programming languages. You can even call an HTTP API from something as simple as batch files or Bookmarks/Favorites/Hyperlinks/Shortcuts. So you don't have to be a hard-core programmer to gain the benefits of Awasu functionality exposed via an HTTP API.

2) It's already present in Awasu. There are already HTTP APIs to start the New Channel wizard, add a web page's title/URL to the Default Workpad and perform several other functions, so this is really an expansion of some existing functionality, not something totally new that has to be totally built from scratch.

Try one for yourself, click on this link to add Awasu's home page to the default Workpad:
http://localhost:2604/workpad?url=http://www.awasu.com

I'll provide more details on the "Run Channel Reports" and "Search Awasu feed items" in subsequent posts.

The sandman cometh. :sleepy:


Sun Aug 30, 2009 3:35 am
Profile
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2897
Location: Melbourne, Australia
Post Re: HTTP APIs
kevotheclone wrote:
some of which he said he no chance of getting implemented

Hey, don't let that stop you from posting a request for them here. If there's a lot of demand for something, I''ll certainly try to implement it :whip:

kevotheclone wrote:
ability to run Channel Reports

This is fairly straight-foward - I'll add something that lets you specify a report GUID, or if it's not recognized, a name.

kevotheclone wrote:
the ability to issue a search in your feed items

This is a little less straight-forward. Just returning some XML with <u>all</u> the matching feed items is easy but paginated HTML results, like Awasu's existing search function is a little trickier. I'll post something here shortly with some ideas...


Sun Aug 30, 2009 4:33 am
Profile WWW
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2897
Location: Melbourne, Australia
Post Re: HTTP APIs
How about something like this:

kevotheclone wrote:
ability to run Channel Reports

Code:
http://localhost:2604/reports/run?id=...


kevotheclone wrote:
the ability to issue a search in your feed items

Code:
http://localhost:2604/search?q=[search term]

which also takes parameters <tt>p=</tt> (page number), <tt>s=</tt> (pagination size) and <tt>f=</tt> (format: html/xml/json).

You'll also need these:
Code:
http://localhost:2604/channels/list
http://localhost:2604/workpads/list
http://localhost:2604/reports/list

to return details about each of the various objects. They will accept a <tt>f=</tt> parameter to specify the format (xml/html/json) and an <tt>id=</tt> parameter if you only want to know about one.

I'll also standardize the other API entry points to use the same object/verb syntax. Maintaining backwards compatibility, of course. And document the whole lot.

Sigh... :whip:


Mon Aug 31, 2009 6:55 am
Profile WWW
User avatar

Joined: Mon Sep 08, 2008 3:16 pm
Posts: 226
Location: Elk Grove, California
Post Re: HTTP APIs
Those all look very good to me! :bow:
support wrote:
"I'll also standardize the other API entry points to use the same object/verb syntax. Maintaining backwards compatibility, of course. And document the whole lot."

Now that sounds like a lot of work. If you want to leave the existing API entry points as they are I think everybody would understand.

Here's some thoughts on some of the individual HTTP APIs:

/reports/run API
I think I've mentioned this before... there are 4 output options that can be specified when a channel report is run, 1) it can be displayed in Awasu, 2) it can be emailed, 3) it can be saved to a file, and 4) it can be FTP'd. If this API is called from a remote machine, should the display of the report in Awasu be suppressed (provided that the user has specified that the specific report should be displayed in Awasu)? Also would this API just return an HTTP Status Code to the caller: 200=success, 500=error.

/search API
Wow this is a lot more than was initially asking for, but if you can provide all this that's great! ::-): From my Awasu Application Plugin perspective I was just thinking that I could pass some search terms to this API and it would open a new tab with the search results using the current search options. But I think in our emails we also discussed that if this API is called from a remote machine that it'd be nice to have a page of HTML search results passed back to the user. If you can provide XML and JSON output formats that would be really cool. There is a spec out there on the OpenSearch web site specifying response elements for RSS 2.0 and ATOM 1.0 OpenSearch response elements, there's probably nothing like this in a JSON format although FWIW there is the OpenSearch JSON Search Suggestions Request/Response format (I know Wikipedia uses this).

So forget about opening a new tab with search results, pass back the XML/HTM/JSON and we'll open a new window or display it in a <iframe> or whatever is appropriate for our application.

If there are any Awasu users (non-programmers) out there, this is a significant development for you too. We can create a Bookmarklet for you, that you can easily install in your web browser that can search your Awasu feed items for any text that you've selected in the current web page. That may not sound too exciting for you if you're sitting on the same PC where Awasu is running, but if you're on an intranet and you discover some terminology that's related to one of your fields of interest and you're wondering "I wonder if these terms appear in my Awasu feed items"; you can just select the terms click your bookmarklets and receive and answer from the Awasu server right away.

/channels/list API
When we discussed this via email I suggested that you just provide an OPML list via an HTTP API, and I've done some work on transforming an OPML file into a couple of HTML formats using XSLT, I'll have provide an update to you via email so you can "see" what I'm talking about. The only problem with exposing the existing OPML format via an API is that it doesn't contain those special Awasu Channel (Search Channels and Plugin Channels). If you can "extend" OPML with extra attributes, then maybe OPML could be the output format for this API.

But, whether this returns OPML or another format, I'll keep working on my OPML transformation code and I'll try to port it over to this output format too.

/workpads/list API
While this would be cool for easily building a report of a complex Awasu configuration. One of my email suggestions (not really a request just a thought "off the top of my head") was a slight modification to the "Add to Default Workpad" API where you specify a specific Workpad via its name or ID so this would be an "Add to Any Workpad" API.
A user could set up Bookmarklets for several of their favorite Workpads and send the URL/Titles of the current web page to the appropriate Workpad.

Now you've already carved out a big chunk of work for yourself and I don't really know how useful this would be given that Awasu version 2.4 can automatically send new items to any Workpads, so unless a bunch of other user's reply with "yes we need a Add to Any Workpad API", I think you can skip this one. But I did think I should bring this thought out into the open in the forum and let other people think about it.

/reports/list API
This would be cool to get a list of all Channel Reports. It would be really easy in almost any programming language to call this API and build a web page of hyperlinks to call the /reports/run API. You could have a Channel Reports "remote control" page on an intranet site and secure it so that only Awasu administrators could access it. With the /reports/list API this code would not have to be updated as Channel Reports are added or deleted; it would always be the most current list.

All of this is sounds like a lot of work and I know you have other areas you want to work on, so if you have to scale this back a little bit, I think everybody will understand. :clap: :clap: :clap:


Wed Sep 02, 2009 3:00 am
Profile
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2897
Location: Melbourne, Australia
Post Re: HTTP APIs
kevotheclone wrote:
If this API is called from a remote machine, should the display of the report in Awasu be suppressed

I don't think so. What if someone wanted a bookmark(let) or something like on the local machine to force a report to be run? If they don't want the report window to open up in Awasu, they can probably just configure it that way.
<i>EDIT: But if we return the report content, this is less of an issue. Maybe we can just add a parameter to the HTTP request controlling whether or not the window opens in Awasu.</i>

kevotheclone wrote:
Also would this API just return an HTTP Status Code to the caller: 200=success, 500=error.

Hmmm, it would make sense to return the report content itself :cool:, or an error message.

kevotheclone wrote:
I was just thinking that I could pass some search terms to this API and it would open a new tab with the search results using the current search options.

Yes, this would be useful as well and probably not too hard to do.

kevotheclone wrote:
When we discussed this via email I suggested that you just provide an OPML list

Ah yes, I forgot about that. We can probably use OPML for everything, it's infinitely flexible :roll:

kevotheclone wrote:
The only problem with exposing the existing OPML format via an API is that it doesn't contain those special Awasu Channel (Search Channels and Plugin Channels).

Don't quite understand what you're saying here. There's nothing in OPML that prevents these special channels from appearing, Awasu just doesn't include them (by default) when exporting to OPML since no other feed reader will be able to do anything with them.

kevotheclone wrote:
While this would be cool for easily building a report of a complex Awasu configuration.

Nah, this'd be insanely cool for another reason: users would be able to add feed items to a workpad, then launch another program that sucked them out and did something with them.

kevotheclone wrote:
One of my email suggestions (not really a request just a thought "off the top of my head") was a slight modification to the "Add to Default Workpad" API where you specify a specific Workpad via its name or ID so this would be an "Add to Any Workpad" API.

Sigh, it's in already (add <tt>id=...</tt> to the HTTP request).

kevotheclone wrote:
I don't really know how useful this would be given that Awasu version 2.4 can automatically send new items to any Workpads

This API lets any app or web page to add an entry to an Awasu workpad, not just newly-received items.

kevotheclone wrote:
All of this is sounds like a lot of work and I know you have other areas you want to work on, so if you have to scale this back a little bit, I think everybody will understand. :clap: :clap: :clap:

Not really a problem. Most releases have a main area of improvement and 2.4.2 will definitely be all about extensions :-)


Wed Sep 02, 2009 8:01 am
Profile WWW

Joined: Mon Mar 09, 2009 2:29 pm
Posts: 27
Post 
Looks very interesting, can't wait to see more information and examples.


Wed Sep 02, 2009 5:01 pm
Profile WWW
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2897
Location: Melbourne, Australia
Post Re: HTTP APIs
I think you'll like this: http://www.awasu.com/api/

Try not to drool too much, it damages the keyboard :hysterical:


Fri Sep 04, 2009 6:43 am
Profile WWW
User avatar

Joined: Mon Sep 08, 2008 3:16 pm
Posts: 226
Location: Elk Grove, California
Post Re: HTTP APIs
I would have replied sooner, but Taka you were right, drool does indeed damage a keyboard. This is really cool, the Templated Results section, blew my mind, but luckily none of it landed on my keyboard. This is very thorough and goes way beyond what I was originally suggesting, you continue to amaze!

FYI The "$/channels/openItem" API might also be useful for something I've got planned, thanks for including it.

I'll have to review this more thoroughly, but one thing that I'm not sure about in the documentation... is it possible to return HTTP status code and messages and the body of the HTTP response if the caller users the awasu:// protocol? Do you just write it to, and we read it from STDOUT?

:clap: Thanks again for continuing to innovate. :bow:


Sat Sep 05, 2009 3:52 am
Profile
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2897
Location: Melbourne, Australia
Post Re: HTTP APIs
kevotheclone wrote:
the Templated Results section, blew my mind

I have to admit I was quite pleased with myself when I thought of it :-D I was worried I wouldn't be able to explain the idea clearly but obviously the concept got through... :roll:

kevotheclone wrote:
FYI The "$/channels/openItem" API might also be useful for something I've got planned, thanks for including it.

The big problem is how applications can know which item ID to pass through but I think there are enough places where it's available to warrant this call.

kevotheclone wrote:
is it possible to return HTTP status code and messages and the body of the HTTP response if the caller users the awasu:// protocol? Do you just write it to, and we read it from STDOUT?

Ah bugger, I didn't think of that :wall: I'll think of something...


Sat Sep 05, 2009 5:03 am
Profile WWW

Joined: Mon Mar 09, 2009 2:29 pm
Posts: 27
Post 
Hmmm, now my keyboard isn't working. :wink:


Tue Sep 08, 2009 2:14 am
Profile WWW
User avatar

Joined: Mon Sep 08, 2008 3:16 pm
Posts: 226
Location: Elk Grove, California
Post Re: HTTP APIs
Hey Jerry, Code Orange looks pretty cool. I'm not an avid blogger, but I could test a beta for you.

Taka, I don't think you have kill yourself trying to devise a way to return results via the awasu:// protocol. This looks like Asynchronous Pluggable Protocol and admittedly I don't know very much about them, but I do know how to call them using a couple of different languages and I'm not so sure that the typical caller can easily capture any returned results.

With the HTTP protocol there are ways that clients can capture the returned results fairly easily, but then there are also ways that clients can call the HTTP protocol and not receive the results (bookmarks/favorites/shortcuts). But at least there is a well-known way to receive the returned results with HTTP; with the awasu:// protocol there may not be an easy straight-forward way. As long as you clearly document that the returned results are only available using HTTP requests, I think everybody will be happy.

I noticed you left out one of the existing APIs; the ability to run an Application Plugin. That's understandable, as you've got a lot to think about.

When I originally started thinking of functions that could be performed with an on-demand API, I was mainly just thinking about the ability to invoke an Awasu function. For instance, for the Run Reports function I just thought that if you could cause the Channel Report to run that would be good enough; it would be up to the process that ran the Report to know that this Report saves it's output in this file, so I'll watch for that file (if the call is asynchronous) and when it's ready to process I'll process it. Reviewing our previous emails I see that we did mention the Search API could possibly return results to a remote client, so clearly you've taken this a lot further than I ever imagined. Which of course lead to yet another revelation, which will not be televised, but will be a future forum post.
:afro:


Wed Sep 09, 2009 4:16 am
Profile
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2897
Location: Melbourne, Australia
Post Re: HTTP APIs
kevotheclone wrote:
But at least there is a well-known way to receive the returned results with HTTP; with the awasu:// protocol there may not be an easy straight-forward way. As long as you clearly document that the returned results are only available using HTTP requests, I think everybody will be happy.

I'm not sure if there's even a way to return anything at all from an <tt>awasu://</tt> call, I'll have to look into.

To be honest, I can't remember why it was even implemented :oops: IIRC, it pre-dates the <tt>feed://</tt> protocol and I wanted a way to let people subscribe to channels without having to know what port Awasu was listening on i.e. no response was required. The question only came up because you found out about it, I don't think it's documented anywhere :-) I might leave it as a hidden feature :roll: unless someone can think of a good reason why it could be useful...

kevotheclone wrote:
I noticed you left out one of the existing APIs; the ability to run an Application Plugin.

Yah, it's not really part of the API, it's more just a way of routing requests to a plugin. But you're right, it should be at least included in the documentation. Plugin requests are not currently available via an <tt>awasu://</tt> call but they probably should be.


Wed Sep 09, 2009 7:44 am
Profile WWW
User avatar

Joined: Mon Sep 08, 2008 3:16 pm
Posts: 226
Location: Elk Grove, California
Post Re: HTTP APIs
support wrote:
I'm not sure if there's even a way to return anything at all from an <tt>awasu://</tt> call...

Probably not, after all they call it an "asynchronous" pluggable protocol.

support wrote:
...I don't think it's documented anywhere...

I know I just saw it in a old release notice, and what kind of nut, besides myself, looks at old release notices. :lol:
A quick Google search reveals it's mentiond in (at least) these release notices: Awasu 2.2.3.alpha3, Awasu 2.2.3 (beta), Awasu 2.2.4.alpha1, Awasu 2.2.4 (beta), Awasu 2.3.4.alpha1, Awasu 2.3.4 (beta). Again with the old release notices, whadami nuts? :shock:

I think you just need to document that the awasu:// protocol can be used to trigger certain functions to occur, but no results will be returned to the caller.

The /plugins/ API is pretty cool because you can expose various functionality of an Application plugin via ParamString. This can be called from User tools or Bookmarks/Favorites/Shortcuts/HTTP-cleint-programs/etc. I'm going to try to finish up the "Open Search" App. Plugin in the next week; I'll email it to you to review.

kevotheclone's jailhouse confession...

When I first looked at Awasu I thought Channel Hooks were the killer feature. Channel Hooks made feed items actionable. And while this is still true and Channel Hooks are still a killer feature, I've come to value Channel Reports and the Template Processing Engine just as much. In the forum topic "create channel hook for all channels" we discussed being able to define Channel Hooks that could be attached to all Channels at once, but Taka thought that "the performance hit would be somewhere between Really Bad and Truly Horrendous". For what I want to do I don't need to get the feed item data as soon as the update occurs, for me at any point in time (but particularly after an "Update All") I could run a Channel Report on the New or Unread feed items that exports all of the New or Unread feed item data into a format that I could code against in an Application Plugin or a web app.
Note: For some people this may not really return all the new items, if they have new feed items being automatically sent to a Workpad with a Channel Report set to run whenever the Workpad is updated with "Mark items as read" checked.

The untelevised revelation:
Now with the inclusion of these APIs exposed through HTTP I can build a list box of Channel Reports and be able to select one of several Channel Reports to process. With the templated output of the Search API, I suddenly realized that I can even programmatically run a search and return a machine readable format that I can process too! :coolthumb:


Thu Sep 10, 2009 4:05 am
Profile

Joined: Mon Mar 09, 2009 2:29 pm
Posts: 27
Post 
... delayed reaction. Hey Kev, I didn't want you and Taka to be the only ones that get to be impressed with themselves. I'll tell you what though, writing your own xml/rss/atom/html parser isn't fun.


Sat Sep 12, 2009 3:55 pm
Profile WWW

Joined: Mon Mar 09, 2009 2:29 pm
Posts: 27
Post 
Verrry interesting. I found a moment to play around with the API and was able to launch a channel report which ftpd an xml file to a remote site. Then of course I was able access it in the browser, as well as import it into Code Orange. So, so far so good. This of course could be done with a timer but it's even better to be able to launch on demand. Very Cool.

I did have some trouble getting a list of channels though but that maybe due to the 600+ channels I have and that my machine is tired. I was able to get a list of channel reports with no problem.

Looking forward to whats ahead but I think my main purpose is already there. I'm sure in the future I'll find new ways to use it.

keep up the good work...
Now back to trying to recover from a major case of burn out.

Jerry


Fri Oct 09, 2009 9:28 am
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 17 posts ]  Go to page 1, 2  Next

Who is online
Users browsing this forum: No registered users and 2 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to: