|Subject:||Re: [Openstack] [Swift] Where do we go from here?|
|Date:||Aug 14, 2012 7:52:58 pm|
It is cool!
2012/8/15 John Dickinson <me...@not.mn>
Swift has many exciting features coming in the OpenStack Folsom Release this fall, but where do we go from here? What's next for Swift in grizzly?
I've got some ideas. I'd like to mention them and see where you the community will take them. I've written up most of them into quick one- line blueprints in Launchpad. If you'd like to contribute, grab the blueprint and jump in.
- Optimize the "many small writes" workload. Swift actually handles many small concurrent writes very well. However, "many small writes" generally also implies that the cardinality of a single container gets very large. There are two ways this use case can be improved:
- Implement transparent container sharding https://blueprints.launchpad.net/swift/+spec/container-sharding
- Provide better listing traversal abstractions. Listing a few billion objects ten thousand at a time is somewhat impractical.
- Solve globally distributed clusters. How can I have servers in London and servers in San Jose in the same logical swift cluster with three replicas total, but guaranteed to have at least one replica in each cluster? https://blueprints.launchpad.net/swift/+spec/multi-region
- Support a single logical swift cluster with tiers of storage (eg cheap spinning disks and expensive high IOPS SSD arrays). Can, for example, a user choose to have a container and its objects be served from a particular tier of storage? https://blueprints.launchpad.net/swift/+spec/storage-tiers
- Some deployers have implemented metadata searching by intercepting write requests and sending the metadata to another system. Can metadata searching be implemented in swift itself? One possible implementation would be to dynamically generate indexes on the container DB. https://blueprints.launchpad.net/swift/+spec/searchable-metadata
- Support PUTs with unlimited size. Implement server-side large object splitting. https://blueprints.launchpad.net/swift/+spec/large-single-uploads
- Support the full HTTP spec for range requests https://blueprints.launchpad.net/swift/+spec/multi-range-support
- There are a few things that could be done to simplify installation
- Create or refactor existing code into a single swift binary or startup script. Would it be possible, for example, to install swift and run one command with the data drives listed and swift "just works"?
- Build a ring server that automatically discovers devices https://blueprints.launchpad.net/swift/+spec/ring-builder-server
- Provide a simple, intuitive way to test a deployment after install https://blueprints.launchpad.net/swift/+spec/post-deploy-test
- Support concurrent reads to objects to support a read-heavy workload https://blueprints.launchpad.net/swift/+spec/concurrent-reads
If you are in the San Francisco area, we will have a swift meetup on August 30 at Citizen Space SF at 6:30 pm. http://www.meetup.com/openstack/events/77706042/
We will have a swift team meeting on Monday October 1 in #openstack-meeting at 8pm UTC to discuss the plans for swift over the next six months and the sessions for the design summit. If you are interested in participating in swift development, please attend.
If you are a new contributor to swift, please read http://wiki.openstack.org/HowToContribute.