The new persister branch is regularly merged into from the current default branch. Whilst it is correct to say it is currently based off the 1.8 release, that’s only true in the sense of the above. The new persister stores messages in a completely different format from the old, and there is currently no tool to allow upgrading from a released version of Rabbit to the new persister without losing persistent messages.

The issue you ran into when going 1.7 to 1.8 is subtly different. Whilst both use the old persister, both the on-disk format of the messages when they are written to disk, and a database schema changed, again, resulting in no state-maintaining upgrade path. To date we have never produced a tool which can do upgrades maintaining state when database schema or on-disk formats have changed.

You talk about users creating queues. I think that what you want is for all users to use queues which have server-generated names, thus you guarantee they are private, and you want to declare them “exclusive”, which means that when the connection that created the queue disappears, the queue itself (and any bindings to the queue) also automatically get deleted.

I quote this text from http://www.rabbitmq.com/admin-guide.html#access-control:

“Some AMQP operations can create resources with server-generated names. Every user has configure, write and read permissions for such resources. However, the names are strong and not discoverable as part of the protocol, only through management functionality. Therefore these resources are in effect private to the user unless they choose to dilvuge their names to other users.”

Thus I think that if you force users to create server-named queues, you don’t need to grant any privileges to your public user. It’ll need read access to the exchange to create the binding, and it should automatically have write access to create the binding to the private queues. If this doesn’t work please let us know.

The 1.8 semantic changes concern what happens when you *re*declare a queue. Previously, if the queue already existed and you redeclare it, but with different attributes, it would still come back with an OK result. This is misleading because it could lead the user to think that a queue had been created with the specified attributes when in fact it has not. Thus now, you must ensure you redeclare with the same attributes otherwise the redeclaration will fail and close the channel. Full details can be found in the lower half of http://lists.rabbitmq.com/pipermail/rabbitmq-announce/2010-June/000025.html

I am very curious about you managing to get the new persister to crash. I’ve been on holiday for the last week and have not caught up with the rabbitmq-discuss mailing list since getting back, so I have no idea whether you’ve posted everything there too. If not, I’d request you do so, and ideally include the various rabbit logs showing the crash – the new persister just *should not* crash.

One thing that might be happening is that Rabbit is raising flow control, to request that publishers stop sending further messages to Rabbit – even with the new persister this can happen sometimes to allow disks to catch up, but this tends to only be necessary at high data rates. The client must respond with a flow_ok message to the broker to confirm it understands the flow control, and it must then not send any further messages – this is usually handled by the AMQP client library as it just makes any subsequent publishes block – until it receives a further flow control message from the broker, informing it it can resume. Now I notice you’re using a python client, and they have historically not supported flow control, which can lead to Rabbit forcibly disconnecting clients that do not respond appropriately to the flow control messages.

You have a very interesting use case, and there is absolutely nothing about it that shouldn’t work perfectly well with RabbitMQ. I’ll try and catch up with the mailing list early next week but in the mean time, please try and dig through the logs, if you’ve not already done so, to see if you can find some sort of stack trace with the crash in it, if it’s there at all, or whether it’s some other issue.