[warning: long and opinionated]

This is about

https://dev.plone.org/plone/ticket/7826

"Improve Plone's import/export story, investigate dump and reload as alternative migration strategy."

The title says it all: the focus of this project is to enable content migrations between different Plone sites irrespective of their version. To enable this Plone's currently existing import/export capabilities should be leveraged or new functionality/modules added - whatever turns out to fit the bill.

In order to get there, let's start by summarizing the current state with regard to:

1. Content import/export

- 1.1 What we have at the moment in the core

- 1.2 What we have at the moment via add-ons

- 1.3 What we think we need to support

- 1.4 What's missing at the moment

2. On a broader scale we might also want to summarize related use cases and existing add-ons created in support of those. While not the main focus here we should aim for peaceful coexistence at the very least but of course we should be able to do better.

3. As many of us have been dealing with the issue one way or the other we might as well invite individual people to contribute here.

I'll start now by filling in on the above but of course you are all invited to contribute in any way you see fit. I suggest we start by discussing this on this list and depending on feedback I'll then turn this into a project on openplans or whatever we see fit later on.

So to get started here is my initial analysis (which is by no means comprehensive but rather an invitation to you all to fill in your stuff or experiences):

1.1 What we have at the moment in the core:

- Zope's import/export mechnism via ZMI

Problem: fragile; doesn't work across different versions of Zope/Plone; probably hard to improve for what we want here.

Raphael's impression: not really an option to start from

- Archetype's marshalling layer and the Marshall product together with the content type registry.

Raphael's impression: good to get individual AT-based content items in and out as well as mass import/export via FTP and WebDAV

Problems: - not loss-less (e.g., doesn't support workflow history or older versions of versioned content) - limited to AT-based content - not easily extensible, e.g., as far as I can see it is not easy to add support for 3rd-party fields like the DataGridField or Record(s)Field, meaning content types using those cannot be handled without going through major pains.

My impression: it shouldn't be hard to fix the above issues so I consider it an option to work our way from there but there might be better options - see below.

1.2 What we have at the moment via add-ons:

- GSXML

- XMLForrest

- collective.transmogrifier

- CMFFolderExport

Raphael's impression: I consider XMLForrest superceeded by GSXML (but of course feel free to disagree should you think otherwise) In general, I think GSXML would provide an excellent starting point. So while there are some issues with it still (as Stefan points out in his comment on the issue https://dev.plone.org/plone/ticket/7826) I agree with Martijn that collective.transmogrifier together with GSXML might be exactly what we want.

But maybe this is only because I never seriously tried to use CMFFolderExport?

What do others think?

1.3 What we think we need to support

I'd say we should aim for a loss-less, round-trip safe, extensible and customizable content import/export machinery that works across different Plone versions to the extend possible. ;-)

That's harder than it may sound first given that we have things like:

- UIDs - References - Workflow (history) - Versions (history) - Annotations - local permission settings - local role settings - executable ownership - comments - ...

What else?

And related: what should be the minimal Plone version we aim to support: Plone 2.5, 2.1 or 2.0.x even?

1.4 What's missing at the moment?

A concise overview and analysis of what we have today in order to figure out what we need to do still. ;-)

2. Related use cases

- Decoupling content delivery from content production It is my understanding that that's what Entransit is about.

- Support cheap hosting solutions by dumping Plone's content to a format (e.g., static HTML pages) that could be uploaded somewhere else for delivery.

- Importing form/exporting to other systems (other CMSs specifically)

- Interoperate with other standards like JCR/JSR 170

3. People with experience on the matter (in no particular order):

- Sidnei da Silva (Entransit) https://svn.enfoldsystems.com/browse/public/entransit.core/trunk/README.txt

- Ramon Bartl and Stefan Eletzhofer (GSXML) http://svn.plone.org/svn/collective/gsxml/trunk/README.txt

- Martijn Pieters (collective.transmogrifier) http://svn.plone.org/svn/collective/collective.transmogrifier/trunk/README.txt

- Tres Seavers (CMFFolderExport) http://palladion.com/home/tseaver/software/CMFFolderExport/README.txt

- Andreas Jung http://www.zopyx.com/blog/when-the-plone-migration-fails-doing-content-migration-only http://svn-public.zopyx.com/viewvc/zope-projects/scripts/export_old.py?view=markup http://svn-public.zopyx.com/viewvc/zope-projects/scripts/import_old.py?view=markup

- MaxM http://mxm-mad-science.blogspot.com/2008/02/migrating-plone-site-via-json-files-1.html http://mxm-mad-science.blogspot.com/2008/02/migrating-plone-site-via-json-files-2.html

Who else wants to be listed here - aka get involved ;-)

Let your comments come in,

Raphael