View unanswered posts | View active topics

Reply to topic  [ 8 posts ] 
Post Updating channels: question
I wrote some code to watch directories change and decided to monitor my Awasu Logs directory...

I have noticed that rather often for some .rss files -- obviously when a channel updates -- I see a pair of "file modification time changed" events in quick succession, i.e., each of these .rss files seems to get written twice.

I can't identify what characteristic makes it happen for some channels and not others, but perhaps (if what's going on isn't immediately obvious to you) you might have a look to ensure unnecessary "extra" file I/O isn't happening.

Separately, is there some "optimization" going on? I notice that when I update certain channels (manually), the .rss file doesn't seem to get updated, but the channel shows a new Last Updated time.


Tue Oct 28, 2003 6:19 pm
Post Re: Updating channels: question
It looks like maybe Awasu writes a zero-length file (or recreates it, which I assume makes it zero length), then writes out the "real" contents. If this is what's happening, why not just create-write each .rss file -- otherwise it looks like extra I/O.

Maybe modification date/time changes dynamically throughout a file's "creating" lifetime, which would account for the initial zero length; however, I (perhaps naively) expect modification time not to be updated/apply until a file is closed...


Tue Oct 28, 2003 6:20 pm
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2888
Location: Melbourne, Australia
Post Re: Updating channels: question
Allan Wilson wrote:
is there some "optimization" going on? I notice that when I update certain channels (manually), the .rss file doesn't seem to get updated, but the channel shows a new Last Updated time.


If the web server returns an HTTP 304 Not Modified, the .RSS file is not re-written (since it hasn't changed) but the channel is flagged as having been updated.

Allan Wilson wrote:
Maybe modification date/time changes dynamically throughout a file's "creating" lifetime, which would account for the initial zero length; however, I (perhaps naively) expect modification time not to be updated/apply until a file is closed...


You would have to write a little test app to confirm this but I would agree that this is naive :-) For example, what if a file was opened allowing multiple readers/writers. If somebody modified the file but still kept it open, I would want the other processes to see a new modification time.

I'll have a look at the double update of the .RSS file as well. Awasu opens the file and then gets the feed before writing it out i.e. there's a delay between opening the file and writing the feed out. So you may well be seeing two file modified notifications.


Tue Oct 28, 2003 6:25 pm
Profile WWW

Joined: Sat Feb 08, 2003 8:36 pm
Posts: 247
Location: San Francisco, CA -- USA
Post File modification times
I did write a small test. I create a file, delay awhile, write some data into the file, delay, write some more data into the file, delay, then close the file.

Whether I flush buffers after each write or not, I see 2 win32 file-modification-time-changed events when my test file already exists: once on creating the file (evidently it gets set back to zero length) and once when I close the file.

Perhaps the two events per .rss file I see from Awasu are these same two events: write (an existing) .rss file, then close it.

If you can give me more insight into exactly how Awasu handles updating .rss files -- or modification-time updates in general -- I would like to know.

Thanks

Allan


Tue Oct 28, 2003 7:36 pm
Profile
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2888
Location: Melbourne, Australia
Post Re: File modification times
To be honest, I strongly suspect that you are *always* going to see two events, even if there is no delay.

Writing to a file like this is always a two-step process: 1) open the file (and truncate it) and 2) write out the data. There is no such thing as the create-write you mentioned earlier.


Tue Oct 28, 2003 7:46 pm
Profile WWW

Joined: Sat Feb 08, 2003 8:36 pm
Posts: 247
Location: San Francisco, CA -- USA
Post 
I'm not sure about always seeing two -- watching the Logs directory for modification-time changes, sometimes I see two events per file, sometimes one.

By "create-write" (a poor term chosen on the spot) I meant when all data is in a single buffer, basically:

f.write("This is the entire text for the file\n")

The file creation (open-for-write) of course happens before. I wanted to minimize modification-time notifications.


Tue Oct 28, 2003 8:21 pm
Profile
Site Admin
User avatar

Joined: Fri Feb 07, 2003 8:48 am
Posts: 2888
Location: Melbourne, Australia
Post 
abwilson wrote:
I'm not sure about always seeing two -- watching the Logs directory for modification-time changes, sometimes I see two events per file, sometimes one.


Welcome to the wonderful world of Windows programming :-) :roll:

abwilson wrote:
By "create-write" (a poor term chosen on the spot) I meant when all data is in a single buffer


This is what happens:

Code:
    ByteVector rssFeed ;
    try
    {
        // get the latest feed
        doGetFeed( &rssFeed ) ;
    }
    catch( exception& xcptn )
    {
        // save the error
        ofstream saveFile( lastRssFeedFilename() , ios::binary ) ;
        saveFile << xcptn.what() ;
        throw ;
    }
    catch( ... )
    {
        // save the error
        ofstream saveFile( lastRssFeedFilename().c_str() , ios::binary ) ;
        saveFile << "Feed update failed: Unknown error." << endl ;
        throw ;
    }

    // save a copy of the feed
    ofstream saveFile( lastRssFeedFilename().c_str() , ios::binary ) ;
    saveFile.write( (char*)&rssFeed[0] , rssFeed.size() ) ;


So, I was actually wrong when I said there was delay between opening the file and writing out the feed (that's how it used to work).


Wed Oct 29, 2003 6:20 am
Profile WWW

Joined: Sat Feb 08, 2003 8:36 pm
Posts: 247
Location: San Francisco, CA -- USA
Post 
Great -- that's very helpful.

Thanks for the detailed info.


Wed Oct 29, 2003 1:32 pm
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 8 posts ] 

Who is online
Users browsing this forum: No registered users and 1 guest

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to: