Page 1 of 1

More flexibility in the deletion of feed with errors

Posted: Tue Feb 28, 2012 10:53 am
by Lossadil
Hi there,

My Awasu runs with more than 2.500 feeds and, obviously, a lot of them are dead after a few months ...

So, is there any possibility to do this in 2 steps ?

1. create a report with feeds which didn't receive any updates for 1 month (in order to check manually on the website itself what's the matter.)
2. delete automatically those feeds

Other question :

Is there any possibility to configure the "clean up channels" option? Because I don't consider as a big error a feed with only 3 errors since this morning.

Because at the moment, I need to do it manually with this "clean up channels" and it takes a long long time ...

Thanks in advance.

Re: More flexibility in the deletion of feed with errors

Posted: Tue Feb 28, 2012 11:28 am
by support
Lossadil wrote:So, is there any possibility to do this in 2 steps ?

1. create a report with feeds which didn't receive any updates for 1 month (in order to check manually on the website itself what's the matter.)
2. delete automatically those feeds

The threshold for no recent content the Clean-up channels dialog uses can be configured by opening your CONFIG.INI file and changing the NoRecentContentInterval value (number of days).

If you want to identify these channels and delete them programatically, you'll need to write a script that uses the Awasu API. The easiest way would probably be to do this:
(*) iterate through all the .CHANNEL files and identify channels that haven't received new content in a while. Look for the LastNewItemTime value (a time_t value) that indicates when a new item was last received.
(*) Then use the Awasu API to delete those channels.

Lossadil wrote:Is there any possibility to configure the "clean up channels" option? Because I don't consider as a big error a feed with only 3 errors since this morning.

Yes. In your DEBUG.INI file, add the following:

Code: Select all

[Debug]
ErrorModeThreshold = ...

where ... is the number of consecutive update errors a channel must get before it goes into error mode (defaults to 3).

Re: More flexibility in the deletion of feed with errors

Posted: Tue Feb 28, 2012 12:13 pm
by Lossadil
Great thanks ... It works great ...

For script ... good idea, I will test it ...

Re: More flexibility in the deletion of feed with errors

Posted: Mon Mar 12, 2012 7:55 am
by kevotheclone
support wrote:The threshold for no recent content the Clean-up channels dialog uses can be configured by opening your CONFIG.INI file and changing the NoRecentContentInterval value (number of days).

Wow, as much as I've learned about Awasu, I still have a lot to learn.

Sorry Guys I've had this sitting tin my Drafts since Feb. 29"

I've been way too quiet on the Awasu Extension front, so as an exercise for me in Python and the Awasu API I whipped up this little script:

Code: Select all

#-------------------------------------------------------------------------------
# Name:        Awasu Channel Cleanup
# Purpose:     Delete's Channel's
#
# Author:      kevotheclone
#
# Created:     28/02/2012
# Copyright:   (c) kevotheclone 2012
# Licence:     public domain
#-------------------------------------------------------------------------------
#!/usr/bin/env python

import argparse
import os
import urllib2
import win32api

# Constants
dirPath = "M:\Documents\Awasu\MyData\Channels"
awasuApiUrl = "http://localhost:2604/channels/delete?token=ABC123&id="

def getChannelsWithErrors( threshold ):
    errorChannels = []

    for filename in os.listdir( dirPath ) :
        fullPath = os.path.join( dirPath , filename )
        val = int( win32api.GetProfileVal( "State" , "NConsecutiveUpdateErrors" , 0 , fullPath ) )
        chanId = int( win32api.GetProfileVal( "Config" , "LocalChannelId" , 0 , fullPath ) )

        if val > threshold :
            errorChannels.append( { "ChannelFilePath" : fullPath , "ChannelID" : chanId , "NumberOfErrors" : val } )

    return errorChannels

def displayChannelsWithErrors( errorChannels ):
    for channel in errorChannels :
        print "Channel ID: " , channel["ChannelID"] , channel["ChannelFilePath"] , "# of errors: " , channel["NumberOfErrors"]

def deleteChannelsWithErrors( errorChannels ):
    for channel in errorChannels :
        chanId = str( channel["ChannelID"] )
        try:
             response = urllib2.urlopen( awasuApiUrl + chanId )
             win32api.DeleteFile( channel["ChannelFilePath"] )
        except urllib2.URLError, e:
            print e.reason

def main():
    parser.add_argument( "-t" , "--threshold" , type=int , default=4 , \
                         dest="threshold" , action="store" , \
                         help="The positive numeric threshold of tolerable Channel errors" )
    parser.add_argument( "-d" , "--delete" , default=False , dest="delete" , \
                         action="store_true" , help="Switch to delete Channels above the theshold" )
    args = parser.parse_args()

    errorChannels = getChannelsWithErrors( args.threshold )
    displayChannelsWithErrors( errorChannels )
    if args.delete:
        deleteChannelsWithErrors( errorChannels )

if __name__ == '__main__':
    main()

There's a couple of hard coded constants that could be parametrized (one could make a call with the Awasu API to get the Channel Files' paths) and I think you need Python 2.7 or newer for the for argparse library, but it works.

Re: More flexibility in the deletion of feed with errors

Posted: Thu Mar 15, 2012 3:06 am
by support
Very cool, thanks :clap:

One thing I noticed, in deleteChannelsWithErrors(), it's not necessary to delete the .CHANNEL file. For reasons too boring to go into, Awasu doesn't delete this file when a channel is deleted but instead sets a flag, then deletes it the next time it starts up.

Come to think of it, if you wanted to be 100% correct, you should also check this flag when trying to identify candidate channels, so that you don't display channels that have already been deleted in the current session.

Re: More flexibility in the deletion of feed with errors

Posted: Thu Mar 15, 2012 8:03 am
by kevotheclone
support wrote:...so that you don't display channels that have already been deleted in the current session.

That's why I started deleting the .CHANNEL files. I noticed that while the call to the Awasu API worked in deleting the Channel from Awasu, if you immediately ran the script again it would list the same Channels.

I didn't know there was a flag set and that the Channels would be deleted the next time Awasu starts. Once again still more stuff to learn.