The Cost Of Database Independence

I read The Cost Of Database Independence and was not impressed

Yes I agree that the way this particular company were allocating ID's was not very good from a performance point of view. But the article then makes two huge and untenable leaps.

Firstly, that there is not a better way to do this and keep the same level of database independance (ie use no database specific techniques such as sequences). This is just not true. We and I am sure many others have been doing it for years. One option is for each client to grab a block of ID's. The client can then allocate ID's sequentially from the block they are given without needing to talk to the server at all. Tune this by allocating blocks of ID's to clients according to their expected volumes and you could reduce the use of the sequence number table by a huge factor.

I once did this for a system that used a dotted id like an ip address. Each part was allocated by one level so that ID's could be alloocated anywhere and still be unique. If I remember the parts were siteid.serverid.clientid.id each time a server started it took the next serverid for its site. When a client started it got the next clientid from it's server then the client allocated id's sequentially. If it ran out it got the next clientid and started again.

Secondly, the other wrong assumption throughout the post is that you can't have database independance and use database specific features. Again it is just wrong, OJB as one example (for Java, or SQLObject for Python and of course there are many many others) handles sequences correctly for a wide range of databases. We have done it ourselves for Firebird, Postgresql and Mysql in the past. It is not rocket science.

If I think cynically then I am suspicious of anything that encourages us to get the performance by sacrificing database independance. In my working life that has been very valuable to allow flexibility in scale of deployment, in platform for deployment, in switching between business models (shrink wrapped vs ASP vs Open Source) and in protecting us from changes in policy/direction etc of the database supplier. When you have used a database where the supplier got taken over and the new owner dropped the product quickly you set a high value on database independance as a form of business insurance.

BTW the link was from: Andrew Channels Dexter Pinion: Database Independence which I found on Planet Python.

Comments

[Andy Todd]

Fair points, in defending Mark though his client had a big performance problem that they could have removed quite easily with the use of sequences. As you say, they *could* have wrapped this (and all of their other database accesses) in a vendor neutral manner in their database access classes (as Philip Eby suggests in his comment on my post) and made their pain go away. But they didn't.

Perhaps my comment about database independence was a bit strident. Maybe the point should have been that you shouldn't stick to inefficient implementations caused by spurious design constraints or architectural principles set by those not qualified to define them.