Hi everybody,
This is a feature proposal, about remote / concurrent access.
A few words about my story. I started a CMS (Content Management System) project, using
Prevayler for storing content metadata. In this system, the work-in-progress content is stored in a WebDAV server, and a proprietary, RMI-based database / workflow server pulls content from WebDAV when requested to. It stores the published content in a "BLOBStore" and metadata is "Prevayled". When I use the term "Database", it means "something keeping data", but be sure that my system is completely
RDBMS-free.
It appeared quickly that all operations on my database should be modeled with Operation objects, which can cary as many (in, out) parameters as wanted. This was looking like
Prevayler commands, plus an information about read / write effect for providing a mutex when required. In fact,
Prevayler provides an interesting basis for handling concurrent access.
But the problem is that my system is *not* prevalent. As a matter of fact, execution of the Operation pulling WebDAV content relies on something which is not a serialized Operation.
Klaus Wuestefeld's answered me :
> A few quick thoughts:
>
> 1) For the "command pulling different content when
> restoring" problem: once the command pulls the content
> (he can do that during his construction, previous to
> execution), the content should be stored in a variable
> of the command and logged AS PART OF IT. This way, the
> command will always apply the same content when it is
> first executed and every time it is restored. You can
> also expect the actual execute() time to reduce
> drastically.
>
> 2) For "non-impacting" commands: they should not even be
> commands. They should simply be methods you call
> directly on your business objects.
His answer points out the difference between Operations (things done on the server) and Commands (modifications to replay on a prevalent system). In fact, Command objects are made to be used in-process, as my Operation objects are a RMI workaround for handling an unlimited number of parameters, plus mutex on write Operations.
Here is example of an Operation composed of several Commands :
- create a Publication object, with a unique Id (from a counter, with synchronized access). Since the counter access does impact the model, it is made in a
Prevayler Command.
- pull WebDAV content in a directory with a name deducted from Publication object Id. This can take a certain amount of time (depending of network speed, etc, and this is not a
Prevayler Command). The pulling returns a bunch of File-like objects.
- create a
Prevayler Command for adding the File-like objects to the Publication, and add the Publication to the model (to a synchronized Collection). This is done at the end, because we can't be sure of the result depending of I/Os. If it fails, we accept to waste the Publication Id (a java 'long').
In this case, only 2 synchronized methods are required for providing the locking scheme. In fact, I did my best to avoid thinking about complex locks. Serializing all Operations may look like a solution, but, in this special case, people requesting a simple HTML page would be stuck while a 50-Mb site is being pulled from the WebDAV server.
This is what lead me to propose some features :
- add an Operation interface, describing "things being done" (reading, writing) on a
Prevayler model. It could deliver Command objects, or other Operation objects (Composite pattern). The Operation object would also make sense for remote use, simplifying parameter passing.
- add an Executor interface, executing Operation objects. A default implementation (ExecutorImpl) would be provided (I don't like those names, any better idea ?).
- a new Executable interface provides a base to Command and Operation.
- the Operation interface can iterate over Executable, for delivering nested Commands or Operations. In a multithreaded environment, an iterator guarantees that Command objects are stored in the *same* order they are executed. The Executable interface was required for this polymorphic behavior. Providing two iterators (one for Commands, one for Operations) would have lead to unclear situations about execution order.
In fact, this sounds much like "reinventing transactional processing" in a real OO world (with nested transactions, but without rollback support). There is nothing about explicit locks, because I think that is a problem of Essential Difficulty.
Except with the Executable interface,
Prevayler original code shouldn't be impacted.
This is how I see the interfaces :
/**
* Base interface for both Command and Operation
*/
public interface Executable {
Serializable execute(
PrevalentSystem ps ) ;
}
/**
* Good ol'
Prevayler Command
*/
public interface Command extends Executable { }
/**
* Can nest both Operations and Commands,
* in a defined order.
*/
public interface Operation extends Executable {
Serializable execute( PrevalentModel model ) throws Exception ;
boolean hasMoreExecutables() ;
Executable nextExecutable() ;
}
public interface Executor {
LockManager getLockManager() ;
void execute( Executable executable ) throws Exception ;
}
public interface RemoteExecutor extends Remote {
Executable execute( Executable executable )
throws RemoteException ;
}
*.command file clean up request.
I am writing a system for embeded devices.
Prevayler is great for object persistence in that environment because of its simplicity and small footprint, except that there is no automatic mechanism for cleaning up old .command files when a snapshot is taken. So as not to use up system resources. --Dan D.
Dan, could that be turned on and off (default = leave the log files there) with a System property? Will that work for embedded? --
KlausWuestefeld
What are the plans to support JDO?
SchemaEvolution / Versioning Support
I haven't done anything more than run the demos at this point, and I like
Prevayler so far. Thinking ahead, I forsee that
SchemaEvolution will be something I'm going to want to handle in a smart way. I realize that part of
SchemaEvolution is an
EssentialDifficulty, but at least a small part of it is an
AccidentalDifficulty. I don't have any solutions, but I think it would be interesting to see an effort to create some generic code to ease the burden of
SchemaEvolution.
I would be interested in some sort of roll-forward recovery functionality. I've had a bit of experience with
Oracle, so I'll use that as my analogy for now.
When you're recovering an oracle system, you can either do a complete recovery or an incomplete recovery. Complete, of course, brings the system up-to-date by applying all valid backups, redo logs, etc. that were successfully generated before system shutdown. This is how a prevalent system works now on every startup. But with incomplete recovery, you have at least two options:
- Time-based Recovery (you specify a date/time, the system is recovered up to that point in time)
- Cancel-based Recovery (the system applies redo logs one at a time, giving you the opportunity to cancel or continue after each)
There are others, but I think they're all
Oracle-specific (like using SCN's and Log Sequence Numbers). I can see valid uses for
Prevayler analogs to these options, and I don't think either would add very much code. Granted, in a prevalent system the cancel-based recovery would probably be per-Command, which for high-traffic applications could get unwieldy. But it might be useful to add this sort of functionality into the API that applications could take advantage of if they like, and doesn't get in the way if they don't.
I'm actually thinking of this for purposes _other_ than crash-recovery. For example, you might want to be able to pull a copy of your production log directory to an internal box when your manager says that some vp needs to know what something or other looked like three months ago.
- Joel Fouse
I think it would be great if a simple (say without joins) SQL command could be translated in Java code executed as a
Prevayler transaction through JDBC. It would help integrate
Prevayler with third-party tools (like Crystal Report) using SQL as a lowest common denominator. I've written a script language that maps identifiers to either Java fields or methods or
JavaBeans properties and handles java.util.List transparently, so I guess it is possible to do the same with SQL. However, I never heard of any free package doing so...
ArnaudClere