atom feed1 message in com.googlegroups.hypertable-userRe: [hypertable-user] non-key indexin...
FromSent OnAttachments
Doug JuddApr 4, 2008 11:45 am 
Subject:Re: [hypertable-user] non-key indexing? ruby client lib?
From:Doug Judd (d.@zvents.com)
Date:Apr 4, 2008 11:45:41 am
List:com.googlegroups.hypertable-user

Hi Rick,

Hypertable does not support secondary "indexes" in the traditional sense. However, you can use multiple tables to get the same effect. For example, if your application data storage can be modeled as a star schema (see http://en.wikipedia.org/wiki/Star_schema), then you can simulate a foreign key join with a simple two-pass lookup.

For example, let's say you have two tables: Employee and Department. In the relational world, the Employee table would have EmployeeID as the primary key and DepartmentID as a foreign key. A join on the DepartmentID column would allow all the employee information along with the corresponding department information to be returned in a single query. With Hypertable, the way it would work is that you would do a two-pass query. The first pass would look up the employee record. The department ID would be extracted from the record and a second pass would use the department ID to query the Department table.

The obvious next question is, "what about referential integrity?". In other words, if I change an employee's job title and department at the same time, but the system crashes and loses one of the table updates, then the data will be inconsistent. This is true, and if you are building an financial application, don't use Hypertable. However, for many web applications, this rare hiccup can be tolerated. And your application can be designed in such away that the data will eventually become consistent, by implementing retry logic that continues until it succeeds.

Remember, Hypertable was designed for scale. The idea is that tables can scale to billions of rows and petabytes of data. Trying to join two petabyte tables together practically infeasible. That is why table joining is not supported.

As far as Ruby support goes, it is high on the priority list for us. Zvents is a Ruby shop, so it will be one of the first supported high-level languages. It should be available in the 1.0 release sometime over the next few months.

- Doug

On Thu, Apr 3, 2008 at 11:22 PM, Rick <rick@gmail.com> wrote:

Hello -

I've read the docs on the project website, in addition to Doug Judd's InfoQ interview and a few other articles/blog posts about hypertable. I am still a little vague on some basic points.

First, Doug mentions that hypertable provides a "scalable solution to store data indexed by a primary key". What about other non-primary-key fields? Are they indexed? Following the example in the "HowHypertableWorks" Google doc, could I, for instance, efficiently retrieve all Items with the tag "velvet" independent of primary key?

Secondly, is there a ruby client library for hypertable?