Encoded Vector Index

[Guest.Visitor]

Has anyone use encoded vector index? I have been reading a lot of information on the internet regarding the subject and everyone said how great it is to use it but never go into detail as to how to use it. I use SQL interphase to create the EVI on one of the file and then use interactive sql to read the file in the order of the EVI index. The SQL did not use the EVI at all. I check the object authority of the EVI and all the read, write, update authority has been removed and I cannot changed it. Any help would be appreciated.

[Guest.Visitor]

CS, An EVI will be helpful for selecting records. It gives the query optimizer statistical information about key values and their frequency. It can also be used to build a bitmapped index more quickly for key fields that the EVI describes. David Morris

[R.Daugherty]

Rochester is sending high level customer execs home with a bizarre story, that of the EVI query that takes 3 seconds instead of 5 hours. No amount of fine print can shake this luring image from the customer's mind, leaving us poor lil' programmers to break the news to them, however ungently we techies are prone to do. Short and sweet of it, EVI is for relatively static data warehouses where you can roll your own against the likes of EssBase. The IBM white papers are clear on this and quite straightforward. Unfortunately, the simple tradeoffs are just way too unsexy to run by the customers. Time to jack up the hype. EVI is similar to the Rushmore bitmapped indexing technology from Foxpro that scared the wits out of Microsoft when it threatened to leave Microsoft and Access in the dust. Time to buy another competitor and bury them, and Microsoft did. The technology is now in the Jet engine that Access and Foxpro share. Nice of Microsoft to sort of let Foxpro still exist. The tradeoff is that EVI indexes are 1/16 the size of an AS/400 logical index over the same fields but cannot be used for sequencing or grouping. They return results in record order as they are just bitmaps over each record. SQL statements that use SORT BY or GROUP BY clauses cannot use them. In addition, they must be created manually ahead of time with some of the same kind of considerations as in planning multi-dimension data warehouse schema. The AS/400 SQL optimizer does not create EVI indexes as needed to perform queries. Also, any change to the data requires rebuilding the EVI indexes. This is what limits EVI to data warehouses that are updated no more often than daily, if that often. They are useful for fields in a static warehouse that are not unique and would not normally be indexed, but may be accessed in ad hoc queries by those searching for patterns, such as queries for "color = red" AND "day = TUESDAY" (I leave just how big your denormalized transaction record will be or the alternative of how many joins you want to do as an exercise). Indexes over sixteen such fields take up the same space as a normal logical and provide better performance with fields that have few potential values, such as color. In my opinion, the space savings are wiped out by the space required for duplicating your production data while also incurring the data integrity issues of keeping your production data in synch with the data warehouse. EVI is ok for those that have chosen to archive off once a week or month to a standalone data warehouse, but hot companies will demand pattern matches against the latest, not historical gold digging that doesn't include this weeks or this month's transactions. That 3 second query that takes 5 hours without EVI? It was a query over a 2 GB unindexed file, where records were read sequentially by the query to retrieve 252 records out of 2 billion that matched the criteria. No comparison was given to performing the query with logicals over the same fields, nor space tradeoffs of logicals versus EVIs, nor the limitation of no SORT BY or GROUP BY in any of the queries, nor the limitation of update versus rebuilding all the EVI indexes. Just 3 seconds versus 5 hours. Marketing does have a way of nutshelling things, don't they? :) Ralph ralph@ee.net

[J.Pluta]

"Marketing does have a way of nutshelling things, don't they? :)" <ROFL!> Ralph, we may not always agree on things, but you do have an ability to turn a sentence. Bravo on an excellent synopsis of EVI. I especially found the Foxpro allusion enlightening... thanks! Joe

[R.Daugherty]

"Marketing does have a way of nutshelling things, don't they? :)" Joe wrote: "<ROFL!> Ralph, we may not always agree on things, but you do have an ability to turn a sentence. Bravo on an excellent synopsis of EVI. I especially found the Foxpro allusion enlightening... thanks!" You're most welcome, Joe. I had to write about something, you said all there was to say about Windows vs. AS/400 web serving.... :) Ralph

[Guest.Visitor]

"Marketing does have a way of nutshelling things, don't they? :)" Joe wrote: "<ROFL!> Ralph, we may not always agree on things, but you do have an ability to turn a sentence. Bravo on an excellent synopsis of EVI. I especially found the Foxpro allusion enlightening... thanks!" You're most welcome, Joe. I had to write about something, you said all there was to say about Windows vs. AS/400 web serving.... :) Ralph Thanks for the great insight. Now I know EVI is not exactly what I am looking for. No wonder why this is not extensively use by others.

[Guest.Visitor]

Let me also suggest you read the following two articles for more information on both EVI and Extended Adaptive Cache which is also a technology somewhat similar to EVI. EVIs: Rapid Response Unit - February 2000, Midrange Computing and... <A > Href="http://www.midrangecomputing.com/mc/article.cfm?titleid=a371&md=200010" Extended Adaptive Cache: The Future of Disk Access Technology</A>