[NBLUG/talk] Multiple writes to the same file.

Thu Dec 23 15:45:59 PST 2004

You've given me some good places to start looking. It's unlikely I'll be
able to do more work on this until next year though, and they've just told
me that they want me to make my daemon into three daemons, simple to say,
but more difficult to do. (it responds to service commands and everything)

It's an online attack system across a VPN. Product is released on a
central mainframe each night at a given time. Each business can obtain a
product through a few simple commands on the mainframe. Naturally all the
businesses are competing with one another for the released products. So
they each obtain multiple logins and script them. The operators of the
mainframe know all about this and are well braced for this nightly event.
Now with the system at idle one instance of my program runs at 9 cycles
per second. During the nightly release I've seen it slow to 75 seconds for
one response from the server.

The server is a IMS system that is upgraded every two years or so. I think
it's made by IBM although I'm not certain.

Now the thing is that when one of my processes grabs a product, it is gone
and no other process will be able to grab that product. Currently I have
that process remove the product from it's hit list. The hit list is really
just an array of product numbers. So it would be very, very good if the
process that grabbed the product could let the other 49 processes know
that it's gone and to remove it from their hit list.

I was thinking that this would be best done through a udp broadcast on a
port that all processes watch, but currently I can only make one process
actually get the message even if several are watching it. It looks like to
me that the $sock->recv($newmsg, $MAXLEN) command grabs the message and
then clears the buffer before the another process has a chance to read the
message (it's nonblocking so the process doesn't actually wait for a
message).

Another design that I was thinking of has a central master process that
simply maintains the hit list and cycles through it always giving the next
product number. I worry though that this could slow down the overall speed
of the processes. What if two request at the same time? etc... If that did
work I'd only have to have it accept input from the fifty processes in the
form of a request or a request with a product number to remove and then
reply with the new product number.

During a idle system time running all 50 processes slows system response
down from 9 cycles per sec to 2.6 so the master process would not have a
lot of time to reply without slowing things down. However in the above
situation cpu usage only goes to 70% or so.

Well that's what I'm working on anyway. Last night I got 2 out of 3
products released, but they weren't very good so I don't think everyone
was really trying for them.

What a long sad, programmers tale it is. I was originally thinking of
moving to c from perl, but as my processes are waiting for the server I
doubt there would be much of a change in speed. I might try to go from
NET::Telenet to raw sockets to bring my memory footprint down a little
though.

> On Thu, Dec 23, 2004 at 11:38:32AM -0800, Walter Hansen wrote:
>> Very good. Now I just need to work out a method of communication between
>> the 50 processes. I did some socket work, but in my experiments sending
>> a
>> message to socket 1000 gets read by one of the proceses and not all.
>> This
>> works great to control my daemon, but the 50 need to be able to send a
>> message that all of them can read.
>
> Sounds to me like you need POE and you just don't know it yet.
> http://poe.perl.org/
>
>> I did just stumble accross IPC::Shareable which sets up special
>> varriables
>> that multiple processes can read to and write from. I'm thinking this
>> might be the 'way' as then I can set up one array that each process
>> could
>> remove elements from on the fly and the other processes would be working
>> with the same data. I do wonder what would happen if one process removes
>> the last element and another process tries to access it (after all, it
>> was
>> there a .00002 sec ago). Race condition? Oh, well. I'll read more. I
>> don't
>> expect to have this up and running this year and the current 50 process
>> works pretty well.
>
> Sounds to me like what you're trying to implement is the classic
> "Worker Pool" multi-threaded/multi-process setup.  You have "n" jobs to do
> and want to run "m" workers at once to process them (where there's more
> jobs
> than workers).  You also sound like either your workers or the controller
> can discover new work to be done, which needs to be put onto the job queue
> for the next available worker to handle.
>
> This kind of programming is tricky to figure out the first time around.
> There's all sorts of subtle little issues you can run into, like
> deadlocks.
> Might be better to find working examples or modules that do most of the
> work
> for you.
>
> In order to avoid the race condition, you need to have some kind of
> lock/semaphore.  IPC::Shareable has lock and unlock mechanisms.  You'd
> need
> to lock the array before reading and unlock when you're totally done.
> (shared read locks don't make sense with perl variables, since a read can
> cause a write)
>
> Also, take into account that you can simplify things significantly by
> having
> a "controller".  Instead of having every process talk to every other
> process, have every process talk to the master controller and have the
> master controller talk to the individual processes.  Then you could use
> IPC::Open3 and set it up so that the master has a pair of file handles for
> each child.  (but each child only has STDOUT to send messages to the
> master
> and STDIN to receive messages)  You'd need the children to communicate
> their
> status and the master to track what they're each doing.  "RUN 1234 param1"
> "STARTED 1234", "ADD 5678 param2" "ADD 9012 param3" "FINISHED 1234" "RUN
> 3456 param4", etc...  You'd need to carefully avoid blocking I/O in the
> master, perhaps by using select.  It's okay if the children block.
>
> Or you could use POE::Component::JobQueue ...
> http://search.cpan.org/~rcaputo/POE-Component-JobQueue-0.5402/JobQueue.pm
>
> How you'd use POE depends on some details, though.  POE is actually
> cooperative multitasking implemented in a single perl process, where I/O
> that would normally block turns into nonblocking I/O that checks if any
> other threads have something to do.  If your workers really truly need to
> be
> separately forked processes, check out POE::Wheel::Run
> (http://search.cpan.org/~rcaputo/POE-0.3003/lib/POE/Wheel/Run.pm).
>
> BTW, POE rocks.  :)
>
>> Oh, hey, Eric it's Walter from UNIX 50A/B back in 93? Hope your doing
>> well!
>
> Oh, you're *that* Walter.  Somehow I'd had it in my head that you were a
> different Walter that I knew about a decade ago.  (no, really, I did!)
> I'm
> doing alright.
> --
> Eric Eisenhart
> NBLUG Co-Founder & Scribe
> The North Bay Linux Users Group
> http://nblug.org/
> eric at nblug.org, IRC: Freiheit at freenode, AIM: falschfreiheit, ICQ: 48217244
> _______________________________________________
> talk mailing list
> talk at nblug.org
> http://nblug.org/cgi-bin/mailman/listinfo/talk
>