Buu Nguyen's Blog

I think Mr. Adzic must have met some real bad MCSD.NET who passed all 5 exams solely by learning by heart the damn brain-dump stuffs from Test King or the similar when saying that he would automatically fail a candidate if s/he happens to have the word MCSD.NET in his/her CV. I do have an MCSD.NET and I think that is way too big a generalization that Mr. Adzic makes.
Read more…

In Part 1 of this article, I have discussed about the Object-Relational (O-R) impedance mismatch, the problem it causes as well as the pitfalls of some O-R/M tools.  In this part, I will examine Object Database Management System (ODBMS) and compare it with Relational Database Management System (RDBMS).

1. What is ODBMS

Basically, an ODBMS is a DBMS which stores objects as opposed to rows or tuples in respectively a SQL or a RDBMS [Wikipedia, Object Database] and bornt out of the need for transparent and non-intrusive persistence of complex object model, tasks which could not easily be addressed by RDBMS because of the O-R impedance mismatch.  As a DBMS, besides being a data respository for storing object graphs (together with their identities, attributes, associations, and inheritance information), an ODBMS would, at the very least, include a query engine, a concurrency management system, and a data recovery mechanism.  (The very first effort to define the features of ODBMS was the ODBMS Manifesto first published in 1989 by Malcolm Atkinson et al.)

Standardization

Before examing the types of ODBMS, it’s worth to learn about the ODMG (Object Data Management Group), which is a standardization committee established in 1991 with the goal of promoting the adoption of ODBMS via the creation of standardized ODBMS specifications.  In 1999, the latest version of the ODMG specification (3.0) was released with the four major components:

• Object Model: defines the common data model (which is a common denominator for OO database systems and programming languages) to be supported by all ODMG-compliant ODBMSs.  With this common data model, object definitions within object databases can be portable among different applications, programming languages, and platforms
• Object Specification Languages: include Object Defition Language (ODL) and Object Interchange Format (OIF).  ODL is used to define the database’ object schema and is equivalent to the Data Definition Language (DDL) in the relational world.  On the other hand, OIF is a means to dump and load the object databases’ state to and from files (e.g. XML files) (e.g. to support the exchanging of objects between different object databases)
• Object Query Language: is a query language based on SQL 92 and is equivalent to SQL in the relational world.  OQL supports the querying of complex objects, polymorphism and late-binding calls, and is interoperable with specific language bindings
• Language Bindings: written for C++, Smalltalk, and Java and expose a persistence API so that these languages can interact with ODMG-compliant object databases

Despites this standardization effort, as of 2001, there was no ODBMS fully compliant to all ODMG standards [Barry, 2001].  In this same year, the ODMG disbanded as the member companies decided to concentrate their effort on the Java Data Objects specification, which was resulted from the ODMG Java Language Binding submitted to the Java Community Process.  In 2006, the Object Management Group (OMG) announced that they would develop a new specifications on the ODMG 3.0 specification and has yet to release any specification since then.  As a result, while many standards (including SQL and the mathematics-based relational model) have been consistently adopted by virtually all RDBMS vendors, widely adopted ODBMS standards simply do not exist yet.

2. Types of ODBMS

Depending on how an ODBMS implementation chooses to persist objects, there are two types of ODBMS: non-native and native.

a. Non-native ODBMS

In a non-native object database, there are two separate object models, one of the application and the other of the database itself.  ODMG-compliant ODBMSs are examples of non-native ODBMSs since they require a separate data schema to be defined, regardless of the existance of the application object model.  In order to query or persist objects from and to non-native object databases, the mapping between these two distinct models must be performed.  For ODMG-compliant databases, the schema is defined by the ODL and the application object model can either be generated from that schema or manually written by developers and then modified by a source-code or bytecode/CIL enhancers (as part of the persistent API, such as JDO, for that particular database implementation) to add persistent behaviors (e.g. to make the class an Active Record) and information (e.g. mapping information).

While the separation between the application object model from the database object model gives non-native ODBMS the advantage of having its databases portable across applications, programming languages, and platforms, it is also the source of problems because application developers have to maintain both of these models as the application evolves.

b. Native ODBMS

In native object databases, objects are stored exactly as they are, without the need to map them into a different object model supported by the databases and vice versa.  In other words, in the world of native object databases, there is just one single object model: the application object model and thus, unlike non-native ODBMS, no ODL and common object model are necessary.  (Note that while no new object model is required, it does not mean that a native ODBMS cannot have its proprietary data format to represent the application object models in the data store.)

The interesting thing is that one can easily implement a simple non-native ODBMS in Java, Ruby or a .NET language using the built-in serialization mechanism which can serialize objects into byte-stream, which can then be stored into a file or sent over the network, and deserialize objects from the same byte-stream.  With the serialization infrastructure, no extra work is necessary for storing objects’ attributes, associations, and inheritance information, and thus one will only need to add an object identification mechanism (e.g. assign an OID field to each object, either hand-coding or, more sophisticated, using bytecode/CIL enhancement [no need in Ruby thanks to its "open-class" feature]), a query API (e.g. query-by-example) and a simple concurrency system (assume the database is shared by just one application at a time, the built-in thread locking mechanism is sufficient) in order to have an ODBMS implementation ².

In contrast with its non-native counterpart, native ODMBS while simplifies the querying and persistence of application object model to the minimum, its databases are not easily portable across applications, programming languages and platforms.  In fact, for two or more applications to make use of the same database, they must have the exact same persistence classes bundled with them (same name, same package/namespace, and attributes with their types).  It is even harder for applications written in different languages to share the same database file because of the differences in naming conventions and base types (framework classes) ³.

3. ODBMS Versus RDBMS

Having looked at the basic features and types of ODBMS, let’s examine its advantages and disadvantages in comparison to RDBMS

a. Advantages

• Rich domain model: since ODBMS can store objects at any level of granularity and has built-in suport for identity, association, and inheritance, OO developers can model their domain classes as richly and expressively as they want without being constraint as with the relational world
• Maintainability: since the application object model and database object model are closely related to each other (in non-native object databases) or even are the exact same model (in native object databa
  ses), it takes less effort to maintain these models as the application evolves
• Development effort: the ability of developers to implement rich domain domain with the least maintainance effort would result in a significant reduction in development time and cost.
• Performance: ODBMS is supposed to perform much better than its RDBMS counterpart, regardless of whether O-R/M tools are used or not, in systems with highly complex object model, since no complex queries (e.g. joins) and mapping are required

b. Disadvantages

• Portability: data in RDBMS can be shared by applications written in any paradigm and platform while ODBMS is tied to the OO world.  The situation is even worse for native object databases since the data cannot be used by multiple applications with different domain classes, even if they share the same data.  As a matter of fact, the clear separation between the relational model and object model further assists portability of RDBMS since these two can evolve independent of each other
• Legacy applications: there are so many applications built with a RDBMS back-end that it is impractical to migrate all these data into ODBMS.  In addition, not only the data has to be migrated, the applications which consume the data will also need to be modified to make use of a OO data access mechanism
• Maturity: the ODBMS industry are new (emerged since the 90’s) and thus is far from close to RDBMS world (emerged since the 70’s) in terms of available system vendors (including the compatibility among database systems from different vendors) and tool supports (such as reporting, OLAP, data transformation, and clustering services etc.)

With the above analysis, ODBMS is not so much of a silver-bullet that some people hope for and thus the decision whether to use ODBMS in a project or not must be considered very carefully.  However, once we have done the homework and decided that object databases can be used for our applications, then we can sit down and enjoy the huge productivity gain which cannot be achieved if we stick with relational databases.  In the final part of this article, we will look at the DB4O object database system.  Stay tuned!

Endnotes

¹ The very first effort to specify the features of ODBMS was the work of Malcolm Atkinson and others in 1989 with the ODBMS Manifesto

² This simplistic implementation, besides the lack of functionalities, has several major drawbacks.  First, searching of serialized objects requires all objects to be deserialized into memory first before an in-memory search can occur while deserialization is a extremely costly operation.  Next, databases created by this implementation are not portable across platforms (e.g. Java to .NET) since the proprietary serialization mechanism of the host programming language is used.  Finally, the serialization infrastructure will break down as soon as the object model evolves with new attributes, associations, and data types.  An alternative could serialize objects into a custom XML format so that searching can be performed quickly using XPath, schema evolution and portability can be handled via the XML binding layer; but then, this is not simple anymore and it is usually better to make use an existing ODBMS system, like DB4O, instead

³ While these are hard, they are not impossible in native ODBMS such as DB4O which allows developers to change the default mapping rules in code (although awkward and should be avoided as much as possible), which we will see in Part 3 of the article

References

• Database Systems, Paul Beynon-Davies, Palgrave Macmillan, 2004
• Java Data Object, David Jordan and Craig Russell, O’Reilly, 2003
• [Barry, 2001], ODMG Compliance, Barry et al, Barry & Associates, Inc., 2001
• ODMG 2.0: A Standard for Object Storage, Doug Barry, Component Strategies, 1998
• ODBMS Manifesto, Malcolm Atkinson et al, University of Glasgow, 1995
• ODMG’s website: http://www.odmg.org/
• DB4O’s website: http://www.db4o.com/
• JDO’s website: http://java.sun.com/products/jdo/
• http://en.wikipedia.org/wiki/Object-oriented_database_management_system
• http://en.wikipedia.org/wiki/Java_Data_Objects

In 1975, Frederick P. Brooks wrote one of the most influential books in the software engineering field, The Mythical Man-month.  Within this book, the essay with the same name, The Mythical Man-month, is cited the most by many software managers, but unfortunately, it is much less often read or understood.  Read more…

What Steve Yegge considers the Next Big Language.  Sound like

• Ruby + (Java || C# 1.x/2.0)
• (JRuby || Groovy || Ruby.NET) + good_tools (esp. IDE)
• C# 3.0 && dynamic_typing (not just type inference) && more_syntactic_sugar (return multiple values, object-literal syntax for hashmap etc.)

1. Abstract

Have you ever felt frustrated for having to develop applications whose back-end making use of a Relational Database Management System (RDBMS), such as MS SQL Server, or Oracle?  Do you think it is a pain to write SQL (or stored procedures) to query some data and then manually map the result set to your object model and back?  Great, you have Hibernate, EJB, iBATIS, and Active Record, but do they really really make the work of object-relational mapping (O-R/M) simple enough and completely transparent while imposing no compromises to the richness and expressiveness of the object model?  If O-R/M is such a big problem, why do we not use an Object Database Management System (ODBMS) instead?  And if ODBMS is possible for certain applications that we are developing, which ODBMS implementation can we use at a start?

In this three-part article, I will attempt to provide the answers to all the above questions.  Please note that most of the concepts and tools described in this article will certainly take more than just one or two pages to be fully presented (in fact, 500-page+ books have been written for several of them), thus I will not discuss in dept about any particular concept or tool – instead the aim is to provide a high-level overview of the key points and interested readers are recommended to learn about the specifics via their own research (the References section can serve as a start)

Okay, with that in mind, the contents of the article are organized as follows:

• In Part 1, I will discuss about the object-relational (O-R) impedance mismatch, its consequences, and ORM tools as a rescue
• In Part 2, I will introduce the readers to ODBMS, its benefits, and the reasons why it still cannot replace RDBMS
• In Part 3, I will introduce the readers to DB4O, one of today’s most popular ODBMS implementations

2. The O-R Impedance Mismatch

As OO languages such as Java and C# have become the mainstream programming languages, the O-R impedance mismatch has become among the biggest problems that application developers are facing today.  Of all the troubles caused by the O-R mismatch, the followings are most notorious

• Representation: in the OO world, classes are, inherently, represented in a nested hierarchical structure (i.e. a Customer object consists of many Order objects which in turn consist of many OrderLine objects and so on), in the RDBMS world, things can only be represented flatly in tables (relations) which consists of multiple rows (records, or tuples) and columns (attributes).  In other words, while classes can be represented in any level of granularity, relational schema is limited to only four primitives: the table, the record, the column, and the cell (intersection between a row and a column).  As a result, the richness of the object model is often compromised (inheritance trees are flattened out, associations are simplified or even removed) for the sake of having it easily mapped to the relational model.  The representational difference between the object and relational worlds is the core of all problems
• Object Identity: two objects, despite having the exact same attributes (and even referencing to same nested objects), can be separate entities in the OO world because objects are identified based on their location in memory.  On the other hand, there is no way for the RDBMS to distinguish between the two records with the exact same data.  Imagine two exact same records in the DB are loaded intro a result set and mapped to two distinct objects, when these objects are updated and persisted back to the database, the database cannot distinguish which record the updates should go to.  To resolve this, the concept of primary-key, while not necessary in the object world, is introduced in the relational world to help distinguish records within a table
• Association: while associations can easily be traversed in the OO world using the built-in object referencing mechanism of the host programming language, they are not very straightforward in the flat RDBMS world in which tables can only be linked together using the concept of foreign-key.  To retrieve an associated record in one table for another record in another table, one must use different SQL “join” statements, instead of “object.attribute”.  (To retrieve representation of deeply nested objects, multiple levels of joins are required.)  Finally, while many-to-many relationship (e.g. Singer and Song) can easily be represented in the OO world, you need to have a link table to represent this relationship in the RDBMS world
• Inheritance: although inheritance can easily be modeled in the OO world (e.g. using the extends keyword in Java), it is much harder to be represented in the RDBMS world, which does not have the concept of “table inheritance”.  Thus, several work-arounds are required to represent inheritance, ranging from complete normalization (aka table-per-concrete-class, which has separate independent tables for all sub-classes of an inheritance hierarchy), or complete denormalization (aka table-per-class-family, which has one big table to contain all the possible attributes of all types in an inheritance tree as well as a “discriminator” column to distinguish among the types), to hybrid solutions (such as table-per-class, which represents each class in an inheritance tree by a table with the children tables linking to the parent tables via the foreign-key mechanism)

3. The O-R Impedance Mismatch’s Consequences and ORM Tools

The most obvious consequence of the O-R impedance mismatch is that developers tempt to create simplistic object model so that the mapping between relational data set into objects (and vice versa) can be done in a straight-forward and less error-prone manner.  In fact, it is not hard to see projects in which domain classes and their attributes are simply one-to-one mappings of the database tables and columns respectively.  And while that does help the data mapping task less painful, it means a huge sacrifice to the richness and expressiveness of the domain model and this in turn affects the maintainability and extensibility of the system.  (The discussion about as to why a simplistic object model negatively affects the ability to be evolved of a system [esp. complex system] will be one of the main topic of my future post[s] about Domain-Driven Design.)

As object-oriented developers are crying for the need of rich domain model, numerous ORM tools are bornt to address it.  Ideally, an ORM tool is expected to¹:

1. Make the mapping between the relational database and the object model as simple and transparent as possible
2. Minimize the constraints imposed on the object model and the relational database schema and allow them to evolve as independent as possible

Unfortunately, these two goals, in many cases, contradict with each other: the simpler and more transparent the mapping is, the more constraints required for the object model and the schema and vice versa.  For example, Hibernate takes the Data Mapper approach [Fowler, 2002], and bases on the mapping rules defined by developers to dynamically generate SQL statements required for the mapping.  While this means a simple usage and an almost transparent mapping, it does impose many constraints onto the object model (e.g. requires certain collection interfaces to be used for object associations so that dynamic proxies can be injected at runtime) and the database schema (e.g. to represent inheritance).  Like Hibernate, a particular implementation of the JDO specification² for RDBMS would impose similar constraints onto the object model and relational schema.  On the other hand, iBATIS³ takes a hybrid SQL-map approach which offers a configurable layer of indirection (expressed in SQL and XML) to “map the parameters and results (i.e., the inputs and outputs) of a SQL statement to a class” [Begin et al, 2007].  While iBATIS is very flexible in term of constraints placed onto the model and schema (because developers still take ownership of writing SQL), it requires more work from the application developers than O-R/M solutions like Hibernate.  Next, Active Record, based on the power of the Ruby programming language and implementing the Active Record pattern [Fowler, 2002], while requires the application developers to write the least amount of data persistence code (in comparison with other full-scaled O-R/M solutions such as Hibernate), it does impose a lot of constraints onto the domain model and database schema, especially by many conventions serving as implicit contract between application developers and the framework (so that no XML configuration file or annotation is necessary).  And finally, it’s worth mentioning about the once-considered a silver bullet EJB 2.x, which is not only hard to use but also significantly pollutes the domain model with all kinds of interfaces and conventions.  As a result, until today there is still no O-R/M tool which can completely resolve those two contradictory goals and really make the developers’ lives as easy as they should be…

That goes back to the question that if O-R/M is such a big problem, why do we not use an ODBMS instead?  That will be the topic of the second part of this article, in which I will introduce the readers to the concept of ODBMS, its benefits, as well as the reasons why RDBMS, despite all of the problems it causes to the object world, will still be there to live.

Hai raised a question about the need for QCs in a project team.  Basically he described an situation in which a manager was probed by a developer who believed QCs were not necessary for the project team because developers and BAs could handle all the QC’s tasks, and thus, having the QCs was an unnecessary effort. Read more…

1. The Code is the Design

At university, most of us are taught that the development of a software should go through the following phases: requirement specification, design, construction (or coding), and testing.  By gathering system requirements (e.g. from the clients, market researches etc.), analysts would come up with a bunch of functional and supplementary requirement documents, use case model and specifications during the requirement specification phase.  Read more…

In my opinion, YAGNI is one of the most misleading practices of eXtreme Programming because it can easily be used as an excuse by some designers to create simplistic and rigid design.  Read more…