Allan Wilson

Postby Allan Wilson » Tue Oct 28, 2003 6:19 pm

I wrote some code to watch directories change and decided to monitor my Awasu Logs directory...

I have noticed that rather often for some .rss files -- obviously when a channel updates -- I see a pair of "file modification time changed" events in quick succession, i.e., each of these .rss files seems to get written twice.

I can't identify what characteristic makes it happen for some channels and not others, but perhaps (if what's going on isn't immediately obvious to you) you might have a look to ensure unnecessary "extra" file I/O isn't happening.

Separately, is there some "optimization" going on? I notice that when I update certain channels (manually), the .rss file doesn't seem to get updated, but the channel shows a new Last Updated time.

Allan Wilson

Postby Allan Wilson » Tue Oct 28, 2003 6:20 pm

It looks like maybe Awasu writes a zero-length file (or recreates it, which I assume makes it zero length), then writes out the "real" contents. If this is what's happening, why not just create-write each .rss file -- otherwise it looks like extra I/O.

Maybe modification date/time changes dynamically throughout a file's "creating" lifetime, which would account for the initial zero length; however, I (perhaps naively) expect modification time not to be updated/apply until a file is closed...

User avatar
support
Site Admin
Posts: 2925
Joined: Fri Feb 07, 2003 8:48 am
Location: Melbourne, Australia
Contact:

Postby support » Tue Oct 28, 2003 6:25 pm

Allan Wilson wrote:is there some "optimization" going on? I notice that when I update certain channels (manually), the .rss file doesn't seem to get updated, but the channel shows a new Last Updated time.


If the web server returns an HTTP 304 Not Modified, the .RSS file is not re-written (since it hasn't changed) but the channel is flagged as having been updated.

Allan Wilson wrote:Maybe modification date/time changes dynamically throughout a file's "creating" lifetime, which would account for the initial zero length; however, I (perhaps naively) expect modification time not to be updated/apply until a file is closed...


You would have to write a little test app to confirm this but I would agree that this is naive :-) For example, what if a file was opened allowing multiple readers/writers. If somebody modified the file but still kept it open, I would want the other processes to see a new modification time.

I'll have a look at the double update of the .RSS file as well. Awasu opens the file and then gets the feed before writing it out i.e. there's a delay between opening the file and writing the feed out. So you may well be seeing two file modified notifications.

abwilson
Posts: 247
Joined: Sat Feb 08, 2003 8:36 pm
Location: San Francisco, CA -- USA

Postby abwilson » Tue Oct 28, 2003 7:36 pm

I did write a small test. I create a file, delay awhile, write some data into the file, delay, write some more data into the file, delay, then close the file.

Whether I flush buffers after each write or not, I see 2 win32 file-modification-time-changed events when my test file already exists: once on creating the file (evidently it gets set back to zero length) and once when I close the file.

Perhaps the two events per .rss file I see from Awasu are these same two events: write (an existing) .rss file, then close it.

If you can give me more insight into exactly how Awasu handles updating .rss files -- or modification-time updates in general -- I would like to know.

Thanks

Allan

User avatar
support
Site Admin
Posts: 2925
Joined: Fri Feb 07, 2003 8:48 am
Location: Melbourne, Australia
Contact:

Postby support » Tue Oct 28, 2003 7:46 pm

To be honest, I strongly suspect that you are *always* going to see two events, even if there is no delay.

Writing to a file like this is always a two-step process: 1) open the file (and truncate it) and 2) write out the data. There is no such thing as the create-write you mentioned earlier.

abwilson
Posts: 247
Joined: Sat Feb 08, 2003 8:36 pm
Location: San Francisco, CA -- USA

Postby abwilson » Tue Oct 28, 2003 8:21 pm

I'm not sure about always seeing two -- watching the Logs directory for modification-time changes, sometimes I see two events per file, sometimes one.

By "create-write" (a poor term chosen on the spot) I meant when all data is in a single buffer, basically:

f.write("This is the entire text for the file\n")

The file creation (open-for-write) of course happens before. I wanted to minimize modification-time notifications.

User avatar
support
Site Admin
Posts: 2925
Joined: Fri Feb 07, 2003 8:48 am
Location: Melbourne, Australia
Contact:

Postby support » Wed Oct 29, 2003 6:20 am

abwilson wrote:I'm not sure about always seeing two -- watching the Logs directory for modification-time changes, sometimes I see two events per file, sometimes one.


Welcome to the wonderful world of Windows programming :-) :roll:

abwilson wrote:By "create-write" (a poor term chosen on the spot) I meant when all data is in a single buffer


This is what happens:

Code: Select all

    ByteVector rssFeed ;
    try
    {
        // get the latest feed
        doGetFeed( &rssFeed ) ;
    }
    catch( exception& xcptn )
    {
        // save the error
        ofstream saveFile( lastRssFeedFilename() , ios::binary ) ;
        saveFile << xcptn.what() ;
        throw ;
    }
    catch( ... )
    {
        // save the error
        ofstream saveFile( lastRssFeedFilename().c_str() , ios::binary ) ;
        saveFile << "Feed update failed: Unknown error." << endl ;
        throw ;
    }

    // save a copy of the feed
    ofstream saveFile( lastRssFeedFilename().c_str() , ios::binary ) ;
    saveFile.write( (char*)&rssFeed[0] , rssFeed.size() ) ;


So, I was actually wrong when I said there was delay between opening the file and writing out the feed (that's how it used to work).

abwilson
Posts: 247
Joined: Sat Feb 08, 2003 8:36 pm
Location: San Francisco, CA -- USA

Postby abwilson » Wed Oct 29, 2003 1:32 pm

Great -- that's very helpful.

Thanks for the detailed info.


Return to “Awasu - Extensions”

Who is online

Users browsing this forum: No registered users and 0 guests