tips and techniques for using GemStone/S and Seaside
Ken Treis has post up called Deep SIXX with XMLPullParser about how he and the folks at his company use SIXX to transfer objects between GemStone and Pharo.
Ken ported Antony Blakey’s XMLPullParser from VisualWorks to Squeak in a (successful) attempt to reduce the amount of memory consumed while copying large object graphs using SIXX.
If you are developing in Squeak/Pharo and deploying in GemStone, then you should check out Ken’s work.
Photo by jayhem (Creative Commons).
Longtime GemStone customer Intercontinental Exchange has been named 2009 Exchange of the Year by Energy Risk magazine.
Earlier today, Ramon Leon posted a very good response to a Pat Maddox question How do you handle constraints on persisted collections?:
I’m trying to make the leap from RDBMS/ORM thinking to pure objects,
and am running into some problems. The first is how to handle
constraints on collections, when the objects inside of that collection
are mutable. Quick example, ensuring that only one user exists with a
certain username. It’s trivial to check that a username is available
before adding the user to a UserRepository. If a user can update his
own username though, it’ll bypass any check on the UserRepository and
invalidate that constraint. A couple options…
1. Have a ChangeUsername command that reserves that username in the
repo, tells the user object to change its username, and then tells the
repo to free the original username
2. When user username: is called, publish a UsernameChanged event that
the repository subscribes to. The repo can choose to reject the
change if it would violate the repo’s constraints.
The first is just a bit ugly and procedural. The second requires the
User object to publish events whenever something happens – which is
easy to forget to do, or you’re left with determining which method
calls warrant the publishing of events. Also you’d have to roll back
the change if the event is rejected.
You could handle that stuff by writing some framework code to manage
the tedious stuff…I don’t love either approach though. How do you
guys tackle this sort of problem?
Pat
And I thought that I would reproduce Ramon’s answer here, because I think it helps folks coming from a relational world to start thinking in terms of persistent objects:
The short answer is you don’t.
The longer answer is you’re still thinking relationally; thinking in sets.
OODB’s and RDBM’s are different paradigms and have vastly different strength
and weaknesses. RDBM’s are much better at guaranteeing data integrity
through things such as constraints and keys rather than collections and
pointers, there’s a theoretical foundation for this, but OODB’s have no such
mathematical rigor. This is a weakness of the OODB. There is no one
provable right way to do it.
On the other hand, RDBM’s have to force everything into a single fixed
schema data structure to accomplish this. The problem is this isn’t the
data structure most programs actually use. RDBM’s don’t speak objects, the
primary method programs actually use to model their domain, and every
attempt to bridge that layer by automatically mapping objects to tables
leads to very serious constraints on the object model. It becomes infected
with compromises to performance or compromises to the flexibility of the
model in order to make it fit. Last but not least you’re stuck with a whole
new layer of code to maintain, the mapping layer, that wouldn’t even be
necessary with an OODB and prior to Rails was often more code than the
domain model itself. If you really enjoy programming with objects,
relational databases will never quite fit right, there’s just an impendence
mismatch that as far as I know no one has ever solved cleanly.
So for a query optimizer, declarative constraints and indexing, you have to
do a whole lot more programming. A fair trade to some for the rigor they
require, but for those who choose the OODB, they’re willing to trade the
rigor for the pragmatism of doing a ton less work and actually getting
things done that just work good enough. I can’t tell you how many RDBM’s
I’ve run across that in production that didn’t have a single foreign key in
place or use constraints at all (often for performance reasons) and chug
along just fine making plenty of money for the gold owner. You’re
experienced with Rails, so you’re aware that most Rails apps don’t use
foreign keys at all. Rails treats the db like a big persistent hash table
preferring to keep the logic in the model. This is a very object way of
doing things. It’s not perfect, but it works well enough in practice.
Look at what Rails has to do to try and fake polymorphic collections,
something that is so utterly trivial in an OODBMs that you don’t even
realize it’s anything special to do. Toss a bunch of object into a
collection and commit; done. The flexibility you gain in modeling things
any way you wish using any data structure you want more than makes up for
the things you lose, i.e. simple declarative constraints and a query
optimizer that lets you just slap on indexes after the fact.
With an OODB you have to think more up front and structure you domain in
such a way that your primary use cases are primarily handled by pointer
navigation. If you’re building a blog, you index the posts by the URL in a
dictionary, you put the comments into a collection in the post. Maybe you
hang the posts off the user who wrote them as well. When it’s all setup,
there’s very little if any querying going on. Now OODB’s do support indexed
collections when you need queries but if you’re writing a behavioral app
rather than a reporting engine, more often than not you don’t want queries
you want tree’s of related objects that actually do things.
If you need to use some special high performance custom written collection
to index your model just the way you want it, an OODB lets you, it’s just
another object after all. If you need collections of objects where every
instance could have wildly different data because you allow the user to add
custom fields, the OODB has no problem with this. It doesn’t care about the
structure of your objects. The price you pay for that flexibility is you’re
on your own, no declarative constraints or query optimizer. On the other
hand, you’re not limited in what you want to do. A relational db just can’t
do user generated fields without converting those fields into rows and
losing its queryability and performance.
Sorry, that turned into a longer post than I’d intended and I’m sure you
already know much of if not all of this, but others who are considering
OODBs might find this useful. It’s a different paradigm and requires a
different style of thinking but pragmatically has a lot of benefits if being
productive is the goal.
Anyway, my approach to your particular issue would be to simply make the
name field immutable after being set the first time. A simple guard clause
in the accessor that threw an exception if someone tried to set the field if
it already had a value and the new value was different would protect any
unintended aliasing bugs. After all, an object should encapsulate its own
business rules and if the field were unique I’d have modeled that by hashing
it in a dictionary by that fields value. It’s not the rigorous guarantee
that a constraint on a set is, but objects aren’t about sets, they’re about
instances and in any real program it’s pragmatic enough to get the job done.
Ramon Leon
http://onsmalltalk.com
In the month since the last beta update, I’ve been mostly heads down working on Metacello a version control/configuration managements framework. Metacello isn’t ready for public consumption yet, when it is, you will hear about it here. [Update later that day] Metacello is not a Monticello replacement, it is a Monticello enhancement.
Besides Metacello there have been a handful of bugfixes that are worth packaging up. The most important of which gets rid of potential infinite recursion in ExceptionA>>description.
No GemTools changes. Added GLASS.230-dkh.231 to update list as an alpha release.
GLASS bugfixes and additions:
To update an existing installation to GemStone-dkh.467 and GLASS.230-dkh.231, follow these instructions. If you have recently downloaded an appliance or installed GemStone/S 64 2.3.1, then follow these instructions telling you how to get a recent GemTools client vm.
If you update to GLASS.230-dkh.231 from a version prior to GLASS.230-dkh.231, a Warning dialog will pop up – proceed through the warning.
—–
[1] Photo by dashananda (Creative Commons).
Norbert Hartl just published Easy remote gemstone, an article outlining the steps for setting up a linux host for remote access by the GemTools. Norbert details the steps for either poking holes in your firewall or using an ssh tunnel.
Photo by extratony (Creative Commons).
Andreas Brodbeck published Seaside project is live! a report on his successful use of GLASS and Seaside for a course management tool application.
Andreas shares some of his tips for doing development in Squeak/Pharo and deploying in GemStone/S.
Photo by gorgeoux (Creative Commons).
Norm Green has announced open enrollment for the following course:
Course: GemStone Performance and Advanced System Management
Dates: June 8 – 10, 2009
Location: Beaverton, Oregon, USA
Cost: $2300 per student.
Registration: send email to training at gemstone dot com
Description:
This three-day course builds on the concepts introduced in GemStone’s
Fundamentals of GemStone/S course. Students gain an understanding of
advanced concepts and issues associated with the performance and
tuning of GemStone’s object server. The instruction format consists of
lectures and guided explorations through actual examples providing
each student with real-world experience.
You Should Take This Course If You
Your Recommended Prerequisites For This Course Are
This Course Will Teach You About
Concepts
—–
[1] Photo by Jeff Blucher (Creative Commons).
If you find yourself wishing that you could work with SOAP and GLASS, you don’t need to head out to the shed and drag out the old bucket and squeegee. All you need to do is load up the latest preview GLASS version and you’ll be ready to go!
Before getting started with SOAP, we need to talk a little bit about Hyper. SOAP needs an HTTP client to access services and an HTTP server to provide services. While there are a number of good Smalltalk-based HTTP server implementations floating around out there, there aren’t very many good HTTP Clients!
Fortunately, Bruce Badger has been continuously updating his Hyper HTTP server and last fall he released a version of Hyper with a very nice HTTP Client implementation. In November, I used the Hyper HTTP Client in a port of Masashi Umezawa’s SoapCore and created a GLASS.231 branch. I needed to create a branch, because the new Hyper was incompatible with the version of Swazoo that I was using for Seaside.
Recently I’ve received a number of requests for SOAP support in GLASS, so I bit the bullet last week and tackled the Seaside/Swazoo/Hyper issue. After a couple of days working through various solutions, I decided to ditch Swazoo and write a simple service class for gluing together Hyper and Seaside. See the section on Using Hyper for details.
To be fair, the Swazoo code used in GLASS was several years old (at least). Janko Mivšek has a newer version of Swazoo (Swazoo 2) ported to GLASS, but for SOAP support I need an HTTP Client and the latest Hyper is the ticket. To use Aida with GLASS going forward, you should probably unload the Hyper package, before following Janko’s instructions for running Aida/Web on GLASS.
Back in November, I was able to find 3 SOAP servers in the wild that were still up and running (4s4c, Microsoft Soap Interop Server and EasySoap++ – as of this writing 4s4c appears to be MIA). The servers were built based on the ROUND 2 SOAP Interoperability Tests Specification a (proposed) SOAP service interface that can be implemented by SOAP implementations as part of an interoperability testing process. This page is the best resource for information about the Interoperability Tests and this page has the results of the Interoperability Tests as of November 2001. The tests were arranged by the SOAP Builders Yahoo group and this looks like the message that kicked things off.
I was happy to find a set of servers out there that were still alive, since the servers for the original interop tests built into SoapCore were no longer available! I then built a client and server on top of SoapCore (SoapInteropClient and SoapInteropServiceImpl) and a set of TestCases (SoapInteropTestCase) so that I could test the GLASS SoapCore implementation against the 3 external services and an internal service. By default the test cases for the external services are disabled (see SoapInteropTestCase class>>skipInteropTesting: for information about enabling the tests against the external services).
If you find that you need to use SOAP with a project start by looking at the interop examples. Also read the sections GemStone Processes and GCI and SOAP/HTTP Server Development Tips for information on doing SOAP development in GLASS.
I have not done any work to add transparent persistence to the SOAP server, but I would imagine that the approach used by FastCGISeaside>>answerResponderRoleCheckingLock: and SeasideHTTPService>>answerToCheckingLock: will work for SOAP. The same rules apply:
If anyone is interested in adding transactions to a SOAP server let me know and I’ll whip out a prototype. Right now I assume that the primary interest in SOAP is as a client.
Paul DeBruicker attended the GLASS Workshop in Santa Fe earlier this month and recently Paul announced (beta mailing list subscription required) his desire to port the Spray client from Dolphin to GemStone/S. Spray is a Dolphin-based package written by Steve Waring that adds WSDL support to SoapCore.
Nice timing.
Hernan Wilkinson has offered to work on the project along with Paul and I’m sure if anyone else interested in SOAP/WSDL support for GLASS, that your help would be appreciated. When it comes to web servers, I’m sure that the more players involved in interoperability testing the better the final product. Join and send mail to the beta mailing list if you would like to participate.
“Ditching Swazoo” is not quite as bad as it sounds. Anyone using FastCGI will not be affected and the new version of Hyper looks to be an improved version.
If you use Hyper in production, then I would recommend that you move forward cautiously. I haven’t put the new Hyper under stress for this preview release. I intend to do so before making a beta release. Hopefully the next beta release (non-preview) will be the last beta release, so I plan to subject the system to some heavy load for both FastCGI and Hyper. As it currently stands, Hyper passes the unit tests and I’ve done limited manual testing.
If you are using $GEMSTONE/seaside/bin/startSeaside_Hyper, then you need to edit the file and replace the chunk of code that looks like this:
server := SwazooServer singleton. server initialize. site := (SeasideSite new) name: 'Seaside'; initialize; host: GsSocket getLocalHostName ip: (GsSocket getHostAddressByName: GsSocket getLocalHostName) port: $1; initializeResourceFromUriPattern: '/', 'seaside'. server addSite: site. GsFile gciLogServer: 'Hyper Server started on port ', $1 printString. SwazooServer startSite: 'Seaside'. "does not return, unless there's an error"
with a chunk of code that looks like this:
server := SeasideHTTPService onPort: $1 acceptInForeground: true multiThreading: true. GsFile gciLogServer: 'Hyper Server started on port ', $1 printString. server start. "does not return, unless there's an error"
If you have been using SeasideSite launchSeaside: 9765 to run Hyper using a GemTools Client, then you should use SeasideHTTPService startServiceOnPort: 9765 instead.
Finally don’t skip the sections GemStone Processes and GCI and SOAP/HTTP Server Development Tips, since they contain valuable information on doing Hyper development in a GemTools Client.
When you do a login from a GemTools Client, a separate Gem process is started on the stone’s machine. The client Smalltalk image and the Gem process communicate via GCI using a relatively simple protocol:
The Gem blocks when a command finishes execution. If you execute a command like:
[[ true ] whileTrue: [ (Delay forSeconds: 10) wait ]] fork.
The command will return immediately and the Gem blocks until you execute another GCI command.
For normal development activities, this doesn’t make too much difference, but if you are trying to debug a web server or a SOAP server, then you need be aware that unless you have explicitly given the Gem control (via a GCI call), your Gem threads will be blocked.
A lot of issues can be fleshed out using unit tests, but occasionally you hit an issue that only appears when the system is running as a server. For Seaside applications we’ve arranged for continuations to be used for the remote debugging of application errors.
I have not added continuation-based debugging to the SOAP server (could be done if there is interest), so in order to debug a SOAP server, you’ll need to run your development vm as a server. You need to do the same thing if you want to debug internal server errors for Hyper or FastCGI.
Given that you understand that your Gem threads are blocked:) while your GemTools Client is active and given that you’ve opened a Process Browser you are ready to do server-side development. At the end of this section, I provide instructions for running SOAP/Hyper/FastCGI servers within your development vm.
For the purposes of discussion, I’ll assume that you’ve started a Hyper server and that you’ve navigated to the WAErrorTest page (click on the image for full size view):
Click on the Raise error link and click on the Debug link and the debugger will pop up in the GemTools Client:
The third process involved is the main hyper thread that is listening on port 9765:
So we’ve got three processes of interest: the main accept connection loop for Hyper, a thread forked off by Hyper to service the browser connection, and the a thread forked off by Seaside to service the Seaside request. We’ve got a debugger open on the the third thread and since we signaled an Error, the exception isn’t resumable. If we close the debugger control will be maintained by the client and the Gem execution is blocked.
Once the debugger is closed you can do a couple of things:
If you hit Alt-. while the server has control, control will be returned to the client, however, you can’t determine which process will be interrupted – the process that is running when the vm detects the interrupt will have the debugger brought up on it.
For all of the server variants, internal server errors are logged in the Object Log – commits are needed in the Soap server to make the log entries persistent. Take a look at the object log after running the Soap unit tests, for examples.
The following three sections provide specific information for debugging server code.
The following expression starts an HTTPServer listening on port 8823 that is ready to serve SOAP requests from the ROUND 2 SOAP Interoperability Tests Specification:
SoapInteropServiceImpl registerAllServices. (Delay forMilliseconds: 100) wait.
The Soap service is started in the background. so the Delay is necessary to allow the server thread to make enough progress to be ready to service client requests. This setup is aimed at convenience for running tests. For a production server, you should arrange for the HTTPServer to be started running in the foreground (see SoapHTTPService class>>named:onPort:).
If you are trying to debug a Soap server error, here are a couple of useful methods to set breakpoints in:
The following expression starts an HTTPServer listening for HTTP requests on port 9765:
SeasideHTTPService startServiceOnPort: 9765
If you are trying to debug a Hyper server error, here are a couple of useful methods to set breakpoints in:
For FastCGI you need to edit your Apache (or lighttpd) config to route all requests to a single server. You also need to stop any FastCGI gems that may be running (you can use the Stop Seaside gems command on the Admin… menu). Then run the following expression to start a FastCGI server listening on port 9001 in your development vm:
FSSeasideHandler startUp: 9001.
If you are trying to debug a FastCGI server error, here are a couple of useful methods to set breakpoints in:
A number of commands were added to the GemTools Launcher.
Take a look at the GemTools change log for a complete history of changes since GemStone-dkh.449.
In addition to the Hyper overhaul and addition of SOAP support:
Take a look at the GLASS change log for a complete history of changes since GLASS.230-dkh.216.
To update an existing installation to GemStone-dkh.466 and GLASS.230-dkh.229, follow these instructions. While it is possible to use the Monticello Browser for upating, I am now recommending that you use the Update Client and Update GLASS menu items on the Update… menu to avoid issues that may be introduced by UI changes.
When you update to GLASS.230-dkh.229 a Warning dialog will pop up (due to the changes in the #defaultAction for Warning) – proceed through warning.
—–
[1] Photo by sea turtle (Creative Commons).
James Foster has just announced the availability of the Seaside tutorial Learning Web Development with Seaside, that he has been developing for several months. The tutorial is the basis of his popular Weaving a GLASS Web workshop and his SoCal tour.
The tutorial is being made available under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 License.
If you are interested in having a Seaside tutorial/workshop in your area, please contact James at James dot Foster at GemStone dot com.
Photo by Rob Shenk (Creative Commons).
A couple of weeks ago we published a one-click GemTools 2.4 download. We named the file 2.4 to differentiate it from the 2.3.1 GemTools client download, but we inadvertently included 2.4 gci files in the download – we haven’t released 2.4 yet.
We’ve now made two one-click GemTools preview downloads available on the download page:
If you are using the GemTools 2.4 download and have experienced problems then you should definitely download and use the one-click file that matches your stone version.
—–
[1] Photo by Nessa Land (Creative Commons).
Twitterings
