Priority Queue

[dacust]

Hmmm, sounds interesting. One concern I'd have with your method would be that if there are a lot of 2 entries, and you write them all to the 3 queue, then a "true" 3 comes in, it's now waiting behind all those 2's you just put out there. Then, as you say, if you need to have a max time for a 2 or 1 to wait, if it doesn't get to it, then your process is actually overwhelmed, so pushing the 2's ahead is only holding up 3's. But let's say this is all OK, and it's what you want. You have two weighting criteria: priority and time. I don't think you can really handle this with dataqs. You probably want to read them into a file with the priority and the time arrived. Then use some kind of algorithm to decide what entry to process next. Possible example:
priority*100+secondswait If a priority 3 has been there 1 second, and a priority 1 has been there 2 minutes, 301>220, so the 3 would go first, but by the time the priority 1 has been there 3 minutes and 22 seconds, if a new pri-3 comes in, 301<302, so the pri-1 would go ahead and run. That algorithm doesn't look too good, but you would write something like that to suit your situation. -dan

[Guest.Visitor]

How about this idea... Have 3 data queues, 1, 2, and 3. I'm assuming from your message that 3 is this highest priority... 1. Have your program read data queue 3 until it's empty. 2. Then read data queue 2 and take action. 3. Then go back to 3 and read until it's empty. 4. Read data queue 2. If it's empty read data queue 1. Repeat.

[dacust]

With Chuck's idea, note carefully where he says "until empty" and where he doesn't. When reading 2 or 1, you'd just move ONE record to 3, and then cycle. This doesn't solve the problem where there are so many 3's that it never gets to the 2's, but like I said, that means your process you've set up is running over capacity. But if you think that could happen, my idea above could help. It's more complicated, but it will help level things out if there are bursts of heavy traffic. However, on a system running properly, I'd think Chuck's way should be more than adequate. -dan

[tdaly@sddsystems.com]

Thanks for emphasizing the "until empty" vs. just one... that didn't really sink in the first time I read it. Some more experimenting. Maybe I can dream up some sort of weighting algorithm between the priority and the age of the record, as you suggested. Tom D.

[Guest.Visitor]

Tom, You're on the right track. If you have 3 separate data queues you can tweak the algorithm any way you want without the complexities of writing entries back to the queue. chuck Opinions expressed are not necessarily those of my employer. "Tom Daly" wrote in message news:6b21a23d.3@WebX.WawyahGHajS... > Thanks for emphasizing the "until empty" vs. just one... that didn't really sink in the first time I read it. > > Some more experimenting. Maybe I can dream up some sort of weighting algorithm between the priority and the age of the record, as you suggested. > > > > Tom D.

[Guest.Visitor]

Tom Daly wrote: > I have a program processing transactions off a data queue. I'm > considering methods of prioritizing the transactions: low, normal, > high. My first thought regarding this is to try and read and process an entry only once and not "bubble" up the priorities for subsequent entries. I'd reverse the priority values and make 1 the highest and 3 the lowest. Read the dataq in keyed order. Use the Sbmjob command to do the actual data processing, utilizing the JobPty parameter as appropriate. The 400 then does all of the prioritization for you. Bill

[M.Savino]

Dan, How would seconds-wait be part of the Key ?? If seconds wait = Time put on the Queue minus Time Read (or processed) then you wouldn't know the value until you'd actually read the Queue.... Isn't that too late to be used as a Key Value ?? Or are you suggesting that the entry be written back to the queue again with a new Key or using Chuck's multi-Queue approach ?? I haven't worked with Keyed Queues so I might be blowing smoke here. But if the Lowest Key value always percolates to the top of the Queue and is Always the first value received (similar to doing a *Loval Setll Read Loop) then I think that there's an easier solution. Instead of writing to the Queue using a priority as a Key, why not just use a Time-Stamp as the Key Value. High Priority Transactions get written to the Queue with the actual current date and time. Medium Priority Transactions get written to the queue (using some date/time math) with Current date and time PLUS nn additional time. And Low Priority Transactions get written to the queue (again using some date/time math) with Current date and time PLUS nn++ additional time. IF the RCV Api always gets the lowest key value, I think this would work and you'd avoid having to re-write the entries back to the Queue. Or am I saying the same thing that you're saying ?? Maybe I should just give up Thinking for Lent..... It would be easier than giving up Beer. Mike

[tdaly@sddsystems.com]

I have a program processing transactions off a data queue. I'm considering methods of prioritizing the transactions: low, normal, high. As an experiment I created a keyed data queue with a key length of 1 and using 1,2, and 3 for low, normal, and high priority. The test program first reads with zero wait time all dtaq entries with key eq 3, and processes them. The first time no data is returned, the program then reads with zero wait time all dtaq entries with key eq 2 and writes each one back to the dtaq but with priority now changed to 3. The first time no data is returned for key eq 2 the program then reads with zero wait time all dtaq entries with key eq 1 and writes each one back to the dtaq but with priority now changed to 2. Then repeat, reading all entries w/key eq 1, etc. This seems to work ok. In my test program the "transactions" are just the enqueue time and the original priority. To "process" the transaction it's just sent to a splf. Higher priority entries that are older than more recent lower priority entries do get processed first. A concern is that a flood of high priority messages could hog the process and normal or low priority messages might have an unreasonably long wait before processing. So my question is, is there a better way to do this? Any ideas? Tom D.

[dacust]

Mike, Actually you have some good ideas there. They key point to what I was saying was lost in one detail: I was saying to read the queue, and then write to a data file. Then the program processing could read the records to decide what priority to assign. This is a slightly clumsy way to handle it, but for priority AND time to somehow intertwine, then like you, I'd get lost trying to do it with just data queues. Unless you re-write to the queue, which like you said is not really an attractive option. I'd see it something like this: PGM A hung on data queue, just writes it to a file. Checks to see if PGM B is active, if not active, SBMs PGM B. PGM B just reads the file, looks at priority, time, and whatever else it wants to decide what to process next. Continues until out of records. Optionally, might have a delay, check again, if still no recs, end. -dan