TwoPhaseCommitIsUnnecessary

This is a rough rendering of a page from the old Prevayler wiki. Please see the new wiki for current documentation.

> JoelShprentz by eMail:
> How do prevalent systems control the world?  How do they send email,
> validate credit cards, print pick lists, order goods, etc?

I will answer the question and also show that two-phase-commits, a common feature in DBMSs, are unnecessary. I am not saying they are USELESS, I'm just saying they are unnecessary.

Let's take the eMail sending example.

Objects in a prevalent systems are divided into two big categories:
#BusinessObjects (including Commands/Transactions).
#CommunicationObjects (Anyone suggest a better name?)

Business objects have transparent persistence provided to them by Prevayler. They must be deterministic and therefore cannot depend on anything that accesses external devices (machine clock, files, mouse, keyboard, network, etc) not even indirectly.

Remember transactions are business objects too (Command will be renamed to Transaction in release 2.0). It is a common mistake to query or modify other systems from within Transaction.executeOn().

It is the communication objects that talk to external devices and other systems. It will be a communication object, a DonutOrderEMailSender for example, that will send our eMails, therefore.

> I assume that one goal of Prevayler is to consolidate in  the business
> objects the logic for initiating these actions.  The business objects'
> response to a particular clock tick transaction could be to transmit an
> order for tomorrows donuts.

This business object shall notify listeners. Our DonutOrderEMailSender (communication object) can register as a listener and transmit the donut order eMails.

> The example code shows one way to do this using the Listener
> pattern.  Various X windows register themselves as account or bank
> listeners.  When a bank or account changes, it notifies all listeners.
> What other approaches have been tried?  What is recommended?

Listening and polling are the options we have. Both are OK. When we use listeners, we must take care that the business object's reference to the listening communication object is transient because the latter is probably not serializable. You can see that in the Prevayler bank demo (demo2).

> I expect the result of the task would be passed to the business
> objects as a transaction.

Yes. When the DonutOrderEMailSender receives the notification from the business object it will:
#issue a DonutOrderInitiated transaction to the system;
#send a donut order eMail to the SNMP Mail Server;
#wait for an OK from the server and, finally
#issue a DonutOrderSuccessful transaction to the system.
Then, if the system crashes, the business object will know not to order the donuts again.

This is a very trivial example. I hope all can see how it can be applied to more complex interactions.

> During playback of a log file, we want the business objects to behave
> exactly as they did the first time, but we don't want to send more emails,
> charge a credit card again, or order more donuts.

When the system is recovered, the listener reference, being transient, will be null. The listener will be able to register itself for listening again only after the system is recovered and will not re-order the donuts, therefore. The Prevayler bank demo shows that well: the GUI (communication objects) does not replay all Transactions while the system is recovering.

"OK, but what if the system crashes right after the DonutOrderInitiated transaction (1) but before the DonutOrderSuccessful transaction (4)?" someone will inevitably ask in panic, "When the system recovers, we will not know wether the eMail had already been sent!". @:o

In that case, we have at least four options:
A) Just send another donut eMail order and perhaps eat twice the donuts (the best option by far) @:).

If we cannot afford that, though, we still can:
B) Flood the Prevayler wiki and MailingLists asking for a new feature: two-phase-commits (also known as "Distributed Transactions"). @:p or simply:

C) Perform a query on the SNMP server: "Have we already sent a donut order eMail through you today?" @;) and react accordingly.

Please note how a system, just by providing the right historical queries, can make two-phase-commits unnecessary for its clients! If all systems provided decent historical queries, two-phase-commits would be completely unnecessary.  @8)

If we cannot find an SNMP server with decent queries or two-phase-commits, though, we still have a last resort:
D) Send another eMail order appending the following notice to it: "Please ignore this eMail if you have already received a donut order from us today." @:)

--KlausWuestefeld
"Keep it Simple, Stupid"
I think the system should go for C. I guess for C to work the following sequence must be executed:
#First execute a prevayler transaction which states that something is going to happen and set the status to 'not happened yet';
#Do the distributed interactions (execute all the time-consuming parts like sending the e-mail);
#Finally, execute a Prevayler transaction which will set the status to 'happened'; If the distributed interactions fail, though, the status is changed to 'failed'. You could even keep a log of attempts and their results.
#On crash recovery, check the status of the objects in the business system and if it hasn't happened yet, try it again starting at 2.

Instead of a single transaction involving distributed interactions, two transactions are used, therefore. Assuming that distributed interactions take a very long time compared to the normal execution time of a prevayler transaction, and knowing that Prevayler uses a single write lock, it is undesirable to let a single transaction involved in distributed logic lock the system for a long time.

Step 1 can not be removed because otherwise the check on restart would be unable to know whether something should happen. It is important in case the sequence fails before executing step 3.

--JohanStuyts


Yes. Approach C is the best, in my opinion.

Two disk writes are inevitable in any two-phase commit scheme, I believe.

You had written: "During replay the transaction that sets the status to 'happened' must be able to find the distributed coordinator again and check the status of the transaction."

I erased it because that is unnecessary. The recovering transaction does not have to find any coordinator because, after all, it already knows the distributed interaction has 'happened'.

--KlausWuestefeld



I'm not sure whether your changes will work. The way I read it now is that you commit the distributed transaction before the Prevayler transaction. If the Prevayler transaction gets executed the distributed transaction must have happened.

But what if the distributed transaction successfully commits and the system crashes before the Prevayler transaction is appended to the command log? You have no way of knowing whether a distributed transaction was already started and what its result was. This could result in executing the same distributed transaction multiple times.

That's why I want the Prevayler transaction to commit the distributed transaction. (Hopefully committing the distributed transaction will be fast so the Prevayler is not locked for a long time.) This way the Prevayler transaction will be in the log before the distributed transaction is committed.

In the command log the Prevayler transaction will have a distributed transaction ID.

During replay the Prevayler transaction must connect to the coordinator to either commit the transaction (if it's still valid) or retrieve the status of the earlier commit.

--JohanStuyts


Prevayler transactions should not talk to external systems. They don't (and shouldn't) know wether they are recovering or executing for the first time.

But what if the distributed transaction successfully commits and the system crashes before the Prevayler transaction is appended to the command log?

The CommunicationObjects, who are always in charge of talking to external systems/devices, on recovery, query the system: "are there any pending distributed transactions I have to complete?", and complete them accordingly.

--KlausWuestefeld.


OK. I've finally got it now. Thanks for your patience. --JohanStuyts

@:) --KlausWuestefeld

----