Category Archive

Monty Williams, James Foster, Martin McClure, and myself will be attending the 16th International Smalltalk Joint Conference 2008 put on by ESUG. The conference will be held in in Amsterdam, the Netherlands from August 25-29, 2008. Camp Smalltalk will run August 23-24, 2008.

Martin will be giving a talk entitled “GemStone for Dummies”:

[The] presentation is an introduction to GemStone/S from the viewpoint of the Smalltalk programmer, focusing on the differences from other Smalltalk implementations…

Monty will be giving a short talk on MagLev:

MagLev is a new ruby VM developed by Gemstone…

I’ll be giving another version of “Glass: Share Everything”:

Seaside has been characterized as a “heretical” framework because it breaks many of the widely-accepted “best practices” for web applications, including “share as little state as possible.” With GLASS (GemStone/S, Linux, Apache, Seaside, Smalltalk) GemStone takes this heresy to the next level where “everything is shared” - transparently and persistently….

James will be giving a tutorial “Glass hands on”:

This hands-on tutorial will present Seaside and walk through the process of building an application using GLASS (GemStone, Linux, Apache, Seaside, and Smalltalk). Topics covered include handling user logins, where to put session data vs. application data, building reusable components, styling a web site with CSS, and an introduction to Javascript…

We had an excellent time in Lugano, Switzerland last year and I’m looking forward to another excellent show this year.

Early registration ends on July 15th, so make your plans and register.

For those of you who missed the great conference in Reno this year, you can head over to the STIC site and view the slides for most (if not all) of the presentations.

For the GemStone talks, you can download the slides for my “GLASS: Share Everything” talk or James’ slides from his “Building A Seaside Appplication (with GLASS)” tutorial.

When video and/or audio becomes available, I’ll post another note.

I’ve just published GLASS-dkh.118 (2.2.5) and GLASS.230-dkh.146 (2.3.0 or 1.0beta9). These two versions should be used with GemStone-dkh.279 (Squeak). All three of the versions can be found in the GLASS project on GemSource. If you decide to upgrade to these versions, first load the version on the GemStone-side, commit and logout. Then load GemStone-dkh.279 into your Squeak image and save your Squeak image.

With this update, you will need to do a couple of things to get things to load correctly:

1. Before loading the GLASS package, turn off autoMigrate or evaluate ‘MCPlatformSupport autoMigrate: false.’ The shape and superclass of WAExternalID has been changed in Seaside2.8.2 and the automatic migrate algorithm chokes. As Seaside sessions expire, the old instances will go away.
2. After loading evaluate ‘WAAllTests initialize’ and turn autoMigrate back on.

This release contains the following features:

• Seaside 2.8.2 (Seaside2.8a1-lr.563) merged
• fix bug38366 - blank versions in history list
• fix bug38916: FSSocketServer>>addConnection: and FSSocketServer>>removeConnection: not threadsafe

See GLASS package history for more details.

Between preparations for Smalltalk Solutions and working with Seaside2.9 and 3.0, I’ve been keeping pretty busy. After Smalltalk Solutions, I hope to find time to make another pass at the tools and do a little blogging about GLASS.

I assume that most of you have heard about MagLev which is a product that brings GemStone/S techology to the Ruby world. It is no secret that MagLev leverages a large chunk of the GemStone/S code base, but what hasn’t been emphasized is the fact that we are planning to release a new version (version 3.0) of the GemStone/S 64 product that is based upon the same MagLev vm technology.

Development for the new vm actually began last summer with an effort to speed up the Smalltalk vm by generating native code. Along the way features have been added, like the ability to support the Ruby language:). For Smalltalk execution, it looks like the 3.0 vm will be 1.5 to 2 times faster than the 2.0 vm.

We are shooting at the end of the year for the release of ‘GemStone/S 64 3.0′ and we will have more product details at Smalltalk Solutions.

For Seaside fans, you will be interested to know that I am in the middle of porting Seaside2.9 to version 3.0. I’m down to 4 failed and 15 errors in the Seaside unit tests.

GemStone has received a few requests for training classes and we are close to having enough students to offer classes (we must have at least 3 students per class).

If you are interested in GemStone training in the next 3 months or so, please contact Norm Green (send mail and Norm will get back to you). Course descriptions are shown below.

Read the rest of this entry »

I’ve just published GLASS-dkh.117 (2.2.5) and GLASS.230-dkh.140 (2.3.0). These two versions should be used with GemStone-dkh.276GemStone-dkh.278 (Squeak). All three of the versions can be found in the GLASS project on GemSource. If you decide to upgrade to these versions, first load the version on the GemStone-side, commit and logout. Then load GemStone-dkh.276 into your Squeak image and save your Squeak image.

I know, I know, just minutes after announcing the availability of a new version of the appliance I publish a set of new packages….So shoot me! Actually, I don’t anticipate that you’ll be that interested in this particular set of versions, since you might not be able to tell that I’ve made any changes at all: I’ve simply taken the first step in heading towards getting the OB-Enhancements packages and friends loaded into GLASS.

With these versions I have started using a branch of the OB-Standard package instead of the OG-Standard package, so only folks like Liliana and Otto from Finworks will really be interested.

I haven’t had a chance to try loading OB-Enhancements, yet - I’ll save that for next week!

[Updated 5/10/2008 - use version GemStone-dkh.278 (needed for 2.3) and load OG-AllTests-dkh.14 (in the GemStone Tools project) if you want to run the OmniBrowser tests in GemStone)].

The fabled 2.3 beta release of GemStone/S 64 is finally available for download (contact Monty Williams or send mail for download instructions) .

The beta includes the following features:

1. Primitive support for UTF8 encoding (100x faster than Smalltalk-based algorithm).
2. Primitive support for HTML encoding.
3. Thread local support for GsProcess and continuations. Thread locals are being used in Seaside2.9.
4. Improved ProfMonitor output code moved from GLASS-only to base image.

The beta is shipped with GLASS.230-dkh.130 and GemStone-dkh.270 (both can be found in the GLASS project on GemSource). GLASS.230-dkh.130 is equivalent to GLASS-dkh.114, except for references to the UTF8 and HTML encoding primitives.

A version of the appliance based upon 2.3 beta 1 will be available shortly.

I’ll be giving a technical presentation at Smalltalk Solutions entitled “GLASS: Share Everything”:

Seaside has been characterized as a “heretical” framework because it breaks many of the widely-accepted “best practices” for web applications, including “share as little state as possible.” With GLASS (GemStone/S, Linux, Apache, Seaside, Smalltalk) GemStone takes this heresy to the next level where “everything is shared” - transparently and persistently….

According to the preliminary schedule, I will be talking on June 20th from 9:15am to 10:00am.

James Foster will be giving a tutorial entitled “Building a Seaside Application (with GLASS)”:

This hands-on tutorial will present Seaside and walk through the process of building an application using GLASS (GemStone, Linux, Apache, Seaside, and Smalltalk). Topics covered include handling user logins, where to put session data vs. application data, building reusable components, styling a web site with CSS, and an introduction to Javascript.

James’ tutorial is scheduled for June 19th from 1:30pm to 5:00pm.

The conference will be held June 18-21, 2008 in Reno, Nevada, so if you haven’t already signed up, get busy and register!

I’ve just published GLASS-dkh.114 which should be used with GemStone-dkh.270 (both can be found in the GLASS project on GemSource). Here are some of the highlights:

1. AutoCommit and AutoMigrate features added to GLASS tools. If you ever do something that is truly experimental, you can turn off AutoCommit, but in the majority of cases you should be using AutoCommit. With AutoCommit turned on, the system will perform a commit after every UI action that can change state. In the Code and Monticello browsers that means there are commits after virtually every operation. In a workspace, a commit is performed after the expression is evaluated (doIt, printIt, inspectIt, etc.).
2. Seaside2.8.1 merged in (Seaside2.8a1-lr.541).
3. Refactored the Exception hierarchy for GemStone-specific exceptions.

You can get more information by reading the comments in the package history.

GLASS-dkh.114 can be loaded into version 2.2.5 of GemStone/S 64 (including the just annouced version 2.2.5.1). It can also be loaded into version 1.0beta6 of the appliance.

The 2.2.5.1 release of GemStone/S delayed the work towards releasing a new version of the appliance based on 2.3 beta, but we should have a 2.3 beta build done later today and the new version of appliance should be available early next week.

We just announced the release of GemStone/S 64 version 2.2.5.1.

2.2.5.1 includes a Seaside extent based on GLASS-dkh.114. For our Seaside users we fixed a bug related to remote breakpoints, so that with 2.2.5.1, you can set/debug/resume remote breakpoints.

As I mentioned for 2.2.5, most Seaside users should wait for the 2.3 beta appliance. 2.2.5.1 was created primarily for those Seaside users that need a production release for development as well as deployment.

Yesterday we announced the release of GemStone/S 64 version 2.2.5. Check out the Release Notes for more information.

From a GLASS perspective we fixed only a hand full of bugs. The 2.2.5 release primarily contains features and bugfixes for a commercial customer.

For GLASS users, I recommend that you wait for the 2.3 beta that is scheduled to be available at the end of this week.

If you’ve been keeping score, you are probably wondering what happened to the new version of the appliance that I promised at the beginning of the month.

The primary rationale for releasing a new version of the appliance is to make base image code and new primitives available. I did have a hand full of interesting changes queued up awaiting the release of 2.2.5, but the release of 2.2.5 has been held open for bugfixes that might be required by a commercial customer who is taking 2.2.5 into production. While my changes are interesting, they aren’t interesting enough to warrant a release on their own.

Our current plan is to target 2.3 as a Seaside release. 2.3 will include:

1. support for remote breakpoints
2. a handful of base image bugfixes
3. primitive support for UTF8 encoding
4. any other low hanging fruit

I plan on tackling the UTF8 primitives this week, so we can get a 2.3 beta release including a new appliance drop sooner rather than later.

Based on my past prognostications you should know that for me, “Planning is a form of dreaming.”

I’ve just published GLASS-dkh.108 which should be used with GemStone-dkh.258 (both can be found in the GLASS project on GemSource). Here are some of the highlights:

1. Arranged for GemStone-based Transcript messages to be routed to the Squeak-based Transcript.
2. Added GemToGemAnnouncement based on Announcement package from Lukas for handling Gem to Gem signals.
3. Parallel debugging is feature complete:
   1. Debugged continuations can be ‘resumed’.
   2. Remote breakpoints available (need 2.2.5 final and startup script changes).
   3. Profiling works in mulit-vm environment.
   4. Object log improvements (rewrite and ui adjustments).
   5. Object log integrated with Transcript. Transcript used when development vm attached, Object log used otherwise.
4. Several bugfixes (see package history for more details).

In earlier posts, I have talked about remote debugging. I have decided (for now at least) that Parallel debugging is a better name for what we’re doing.

In a nutshell, Parallel debugging is about providing a set of tools for doing debugging that spans multiple vms that are operating in concert. The goal is to make debugging/development in a system that may be composed of hundreds of vms as easy as doing debugging/development in a single vm. Debugging continuations, remote breakpoints and the object log are all components of Parallel debugging support.

I plan on writing a post that will go into more detail about Parallel Development/Debugging with GLASS.

In my post on Simple Persistence for Seaside, I claimed that with the recent enhancements to the Seaside framework in GLASS, it is perfectly okay to use an unprotected class variable to persist shared state. The use of an unprotected class variable will result in occasional commit conflicts, but when a conflict occurs, the framework does an abort and retries the original HTTP request.

So, just how occasional are these commit conflicts and what if I don’t like the idea of using anything uprotecteced. In the last two posts (Ready… and …Aim…), I introduced the tools that you need to answer the question for yourself. In the end it is the behavior of your application that matters the most.

As targets examples, I have written three very simple Seaside examples. Each of the applications is a variation on the theme of shared counters:

• WATally - counter using a class variable, expect retries due to commit conflicts.
• WARcTally - conflict free RcCounter, no commit conflicts.
• WASerial - use an object lock for the production of a unique value, expect retries due to busy/dirty lock.

WATally

With WATally, when a user clicks on the tally link, the current value of the Tally class variable is incremented and the result is stored back intoTally. This is a sure-fire formula for commit conflicts. The following diagram depicts the timeline for three overlapping tally requests:

Request 2 begins before Request 1 is finished, so its commit fails. After a short delay, the request is retried and succeeds. Request 3 also begins before Request 1 is finished, waits a bit and retries, but its retry attempt starts before Request 2 is finished, so the request fails again. Finally on the second retry, the commit for Request 3 succeeds.

If you run jcrawler against WaTally, you can expect to get a handful of Commit failures in the object log (http://localhost/seaside/tools/objectLogin an appliance). I found that using an interval of 1 is best for scaring up Commit failures. Click on the graphic below to see a full width log entry.

If you want to delve into the causes of the commit failure, you can inspect the object log using the following expression:

    System abortTransaction.
    OTRemoteDebugger objectLog asArray inspect.

In this case you’ll find that the commits failed because of a Write-Write conflict on an ‘Association (#Tally->3864)’, which is expected.

It’s worth noting that jcrawler ran at about 12 requests/second for 2 minutes to generate the 38 ‘Commit failures’ and the retry limit was not exceeded. This is validation that ‘retrying requests on commit failure’ is a perfectly safe technique.

WARcTally

With WARcTally, when a user clicks on the tally link, a shared RcCounter instance is incremented. RcCounter is designed to allow concurrent, conflict free increments and decrements. The following diagram depicts the timeline for three overlapping HTTP requests.

Since RcCounters are desinged to be conflict free, each request completes without the need for retries.

If you run jcrawler against WaRcTally, you can expect an empty object log:).

Reduced conflict classes are still the best way to go, as they are designed to prevent commit failures and have undergone a ton of testing.

WASerialNumber

With WASerialNumber, when a user clicks on the tally link, an object lock is obtained on the shared instance of SerialNumber and the value instance variable is incremented and returned. If the lock is either changed or denied a WARetryHTTPRequest is signalled and the HTTP request is retried, much like the retry after a commit conflict for WATally. Object locks are not transactional, so an object lock can be acquired in the middle of a transaction. In this particular example, the lock (once acquired) is kept for the duration of the transaction.

The following diagram depicts the timeline for three overlapping HTTP requests:

Shortly after the processing for Request 1 begins, the SerialNumber object lock is acquired and when Request 1 finishes processing the lock is released.

Request 2 makes its first attempt to acquire the lock, while the lock is held by Request 1, so the lock request is denied. Request 2 signals a WARetryHTTPReqeuest. On the second retry attempt, Request 2 successfully acquires the lock and completes normally.

Request 3 makes its first attempt to acquire the lock after the lock is released by Request 1, so the lock is granted, however, since the transaction for Requst 3 started during the transaction for Request 1 and the SerialNumber instance is dirtied (the value instance variable has a new value), the lock is granted, but with a changed status. In a generic GemStone/S application, an abort would be performed and processing would continue, however, in GLASS/Seaside aborts are not allowed, so a changed lock is treated the same as a denied lock - a WARetryHTTPReqeuest is signalled. On the second attempt to acquire the lock the request is denied, because the lock is held by Request 2. A WARetryHTTPReqeuest is signalled again. Finally, on the second retry, the the lock is granted and the request is completed.

The GLASS/Seaside framework drops an entry in the object log whenever a WARetryHTTPReqeuest is signalled, so you can see the frequency of failed lock attempts. Click in the following image to see the full width log.

If you look closely at the full-eidth image, you will see the two different kinds of ‘Lock not acquired’ messages. ‘SerialNumber lock changed’ and ‘SerialNumber lock denied.’

jcrawler ran at about 10 requests/second for 2 minutes to generate the 72 ‘Lock not acquired’ entries.

Object locking as a technique for avoiding commit conflicts shares the ‘retry on failure’ model of WATally, so it isn’t necessarily superior to ‘retry on commit failure’. It is a superior technique, if you need to protect logical updates where a physical conflict cannot be guaranteed (i.e., updates to different portions of an object graph).

Summary

It certainly looks like the Simple Persistence model of retrying a request on commit failure will stand up under a moderate load. 10 requests/second without retry failures is a pretty good clip. In a real web site, the potential for conflicts is much lower than in these examples since most web page hits are read only, so you can expect to withstand an even higher overall request rate without too much trouble.

It is still a good idea to use reduced conflict classes or object locks in places where you know that concurrent updates can occur, but I think is important to realize that you you don’t have to protect every shared variable from concurrent updates.

Have fun.

Once you’ve fixed jcrawler so that it can follow dynamic URLs like those used in Seaside, you are ready to aim it at a Seaside application. In this post I’ll give you some pointers for getting started with jcrawler.

Out of the box, jcrawler is written to be run in the jcrawler/dist directory. There are relative paths in the run.sh shell script. That isn’t too convenient, so here’s a bash script that lets you run jcrawler from anywhere:

#!/bin/bash
export JAVA_HOME=<path to java>
export JCRAWLER_HOME=<path to jcrawler>
$JAVA_HOME/bin/java -cp "$JCRAWLER_HOME/:\
$JCRAWLER_HOME/lib/log4j.jar:\
$JCRAWLER_HOME/lib/commons-logging.jar:\
$JCRAWLER_HOME/lib/commons-httpclient.jar:\
$JCRAWLER_HOME/lib/commons-digester.jar:\
$JCRAWLER_HOME/lib/commons-collections.jar:\
$JCRAWLER_HOME/lib/commons-beanutils.jar:\
$JCRAWLER_HOME/dist/jcrawler.jar:\
$JCRAWLER_HOME/lib/htmlparser.jar" com.jcrawler.Main

When you run jcrawler, it expects there to be a crawlerConfig.xml file in a local conf directory (./conf/crawlerConfig.xml). The following is one that I have used for testing against an appliance running at 10.80.21.64 on our internal network. I’ve supplied 4 starting URLs, one for each of the GemStone Examples (randomError, rcTally, tally and serial):

<!--
    Interval (in milliseconds) to invoke a crawl thread.
    There is an HTTP hit every  millisecond.
-->
    <interval>250</interval>

<!--
    Interval (in milliseconds) to invoke a monitor thread.
    Monitor adds new entry in the monitor.log every
    milliseconds
-->
    <monitorInterval>6000</monitorInterval>

<!-- HTTP connection timeout in milliseconds -->
    <connectionTimeout>30000</connectionTimeout>

<!-- Headers to be used by the http client crawler -->
    <headers>
	<header name="User-Agent">Mozilla</header>
	<header name="Cache-Control">no-cache</header>
	<header name="Accept-Language">en-us</header>
    </headers>

<!-- URLs to start crawling from -->
<crawl-urls>
<url>http://10.80.21.64/seaside/examples/GemStone/rcTally/
    </url>
<url>http://10.80.21.64/seaside/examples/GemStone/serial/
    </url>
<url>http://10.80.21.64/seaside/examples/GemStone/tally/
    </url>
<url>http://10.80.21.64/seaside/examples/GemStone/randomError/
    </url>
</crawl-urls>

<!--
    URL patterns (regexps!!!) to allow or deny set of URLs
    permission=true  - these patterns are allowed (anything else is denied)
    permission=false - these patterns are denied (anything else is allowed)
-->
    <url-patterns permission="true">
	<pattern>.*?10.80.21.64.*</pattern>
    </url-patterns>

</settings>

With an interval of 250, jcrawler will run at about 4 requests/second which is fast enough to get started. If you don’t see fireworks at this rate, you can set the interval to zero and let jcrawler fire at will. To really hammer an application you can launch multiple instances of jcrawler.

Here’s a sample object log from one of my runs:

Notice that the report is dominated by entries labeled ‘Lock not acquired - retrying’. A glance at the full width log will show that the retry is due to a ‘Session lock denied: 2075′. A session lock is denied (and the request is retried) if two requests for the same session are received at the same time. This is not surprising given the fact that jcrawler uses a FIFO to store the URLs it scrapes from a page - almost every URL on a single page will have the same session key. When you see errors like this showing up in the object log, you at least know that jcrawler is firing simultaneous requests at Seaside.

As a final note, you should set deployment mode to true (using the Configuration Editor) before pointing jcrawler at your application. If you don’t, you are guaranteed to get some fireworks. While you’re in the Configuration Editor, take a look at the field for setting the Root Component. If the clear link is pressed, the root component for that application is wiped out. Until a new root component is set you will get internal server malfunctions, whenever the WADispatcher tries to launch a new instance of the application. If you don’t set deployment mode to true, jcrawler will eventually find its way into the Configuration Editor and it will eventually hit the clear link.

You are now ready to launch jcrawler at the WATally application and see how our Simple Persistence example fares under load.

Before we point jcrawler at a Seaside application I would like to talk about what you should expect.

To start with, jcrawler is like a bull in a china shop. The algorithm jcrawler uses is not very deterministic nor is it discriminating, but it is thorough, relentless and highly parallel. Given an an initial set of URLs, jcrawler traverses each page and adds the links it finds to its list. Every so often, jcrawler creates a new thread to process another URL from the list. We can depend upon jcrawler to rattle every piece of china in an application and it will rattle more than one piece at a time, so we’d better be ready to deal with wreckage.

Jcrawler will help make your application bullet proof, but at potentially 15 errors/second spread across several vms, there can be a lot of wreckage to sift through.

I added an object log to GLASS a couple of weeks ago and over the last couple of days, I’ve added a Seaside application for viewing and manipulating the object log. Take look at a sample log (my blog is wide-image challenged). It’s not the purtiest page this side of the Mississippi (I am web-design challenged:), but it does the job.

In the object log, the entries labeled ‘– continuation –, represent object log entries that can be debugged via the ‘Debug’ button in the GemStone/S Transcript Window. If you take a peek at the pid column, you will notice that the log entries were generated by two different gems. There are three gems serving HTTP requests in the appliance.

The upshot is that after letting jcrawler hammer on your application, not only do you get an overview of the problems uncovered during the run, but you can open a debugger and investigate the issues that resulted in walkbacks.

I generated the sample object log by manually playing with the randomError application (http://localhost/seaside/examples/GemStone/randomError in the appliance) found in the GemStone Examples project. This little gem generates a simple log entry (’random error’) or walkback (’– continuation –’) 12% of the time. You can also generate different kinds of errors by poking the links in the Error tab of the alltests application (http://localhost/seaside/tests/alltests in the appliance).

If you want to play with the object log, load up the latest version of the GLASS package (GLASS-dkh.103 in the GLASS project - it will also load the GemStone Examples). Poke around in the randErrror application until you get an error then head on over to the object log (http://localhost/seaside/tools/objectLog in the appliance). You can also try the remote debugger from your development image.

Next up we’ll talk a little bit about configuring jcrawler.

The 16th International Smalltalk Joint Conference is taking place August 25-29 in Amsterdam, the Netherlands. Stéphane Ducasse announced the Call for Contributions, so start making your plans to present or be present at the conference today!

The dogs didn’t let us sleep in this morning, so I’m getting an early start on the day. I’ve been very busy the last two weeks intending to write but getting roped into one thing after another.

I’ve just published GLASS-dkh.101 which should be used with GemStone-dkh.253 (both can be found in the GLASS project on GemSource). Here are some of the highlights:

1. GemStone examples included in the GLASS package. I plan to write descriptions of the examples today!
2. Highlighting added to MCRepositoryBrowser.
3. Started work on MCConfiguration support. I’ve got an OmniBrowser-based tool written and MCConfigurations themselves are mostly working.
4. Improved OmniBrowser tools performance. I did some work to reduce roundtrips and speed up icon calculations.
5. Added MCServerDirectoryRepository. Now you can have directory-based repositories on your client machine (where the Squeak client is running) or on the server machine (where your gems are running).
6. Fixed some bugs, etc. You can look at the history for GLASS-dkh.101 if you want the gory details.

Next week we are planning on releasing a new version of the appliance. Keep an eye on this space for an announcement. If you haven’t played with the appliance yet and have 64bit hardware, drop us an email to get download instructions.

It’s an absolutely beautiful day today. Every February we get a week of sunny, fifty degree weather and it looks like this is the week. It is so nice to see the sun! Over the holiday weekend we were able to take our dogs for a run in a nearby dog park. It’s a real treat to see a Saluki in full flight.

Besides playing with dogs, I’ve spent some time playing with and cleaning up some things in GLASS. I’ve published a new version of the GLASS package (GLASS-dkh.88) and here are some of the hightlights:

1. I’m getting happier with the remote debugging factility - the steps for debugging are greatly simplified and I’ve even replaced the Hyper button on the GemStone Transcript with a Debug button that will open a debugger on the first continuation in the RcQueue (you’ll need to load GemStone-dkh.240 into your Squeak image for this feature). When you’re in production (#deploymentMode == true), the debug continuations are unconditionally added to the queue.
2. In the same vein as remote debugging, I’ve added a consolidated object log where you can dump a LogEntry into an RcQueue from any one of the gems serving Seaside requests and view the logged objects in the comfort of your own development image. No need to muck around in N different logs to look for stack traces or important error messages.
3. You may have noticed that in the past I have been somewhat vague whenever the subject of commit failures was brought up. Now, when you hit a commit failure, we drop a transactionConflicts Dictionary into the object log (see the comment in System class transactionConflicts for a description of its contents), and simply retry the HTTP request. If retry fails 10 times in a row, we throw in internal error.

I’ll be writing a couple of posts in the next week or so to provide more details.

Head on over to the STIC site and register for Smalltalk Solutions 2008.

Version 1.0beta6 of the appliance is rolling off the assembly line and that new software smell is once again wafting through the hallways of our Beaverton office. If you are using an appliance that is older than 1.0beta5, then you should contact us to get download instructions for 1.0beta6.

If you haven’t played with the appliance yet and have 64bit hardware, drop us an email to get download instructions, you should also read the Terse Guide to GLASS Tools.

On the development front, I am busy getting a port of SourceSource 2 running on GemStone. SourceSource 2 is Philippe Marschall’s port of SqueakSource to Seaside2.8 and Magritte. When I’m done I think that this will be a good example of what types of things need to be done to port a Squeak Seaside app to GemStone.

Oh, and don’t forget to sign up for our GLASS Beta mailing list.

Yesterday we announced the release of GemStone/S 64 version 2.2.4. Check out the Release Notes for more information.

From a GLASS perspective, there were a handful of changes made to the base product:

• Curly brace Array constructors added to the Smalltalk compiler.
• Support for Squeak-style Pragmas.
• Notification mechanism for abortTransaction, beginTransaction, commitTransaction, and transactionMode:. (see the class SessionMethodTransactionBoundaryPolicy).
• Notification mechanism for sessionStart (see the class SystemLoginNotification).

You need the changes in version 2.2.4 in order to use the GLASS tools. As of today, you can load the Monticello package GLASS-dkh.57.mcz into GemStone/S from the GLASS repository (you will need GemStone-dkh.234.mcz loaded into your Squeak image as well).

In addition to the base support for Monticello and Seaside, GLASS-dkh.57.mcz includes several packages from the SeasideExamples, Scriptaculous, SeasideAsync, Magritte, and Pier. All of the unit tests are passing for the packages. You can look at the version history for GLASS-dkh.57.mcz to get a blow by blow account of the changes.

In the last week before shipping 2.2.4, we found a bug that affects doing a proceed from a halt or breakpoint when debugging Seaside applications. In order to fix the bug, we had to make changes to the C executables. Since 2.2.4 is being released to customers for production applications, we didn’t want to delay 2.2.4, so the bugfix wasn’t included.

After the 2.2.4 release, we did a beta build for version 2.2.5 and included the bugfix. The new version of the appliance (which is in the final phases of production) is being built with the 2.2.5 beta. I recommend that everyone use the 2.2.5 beta for Seaside development.

So you’ve been working with Seaside in Squeak for a little bit and you’ve gotten to the point where you have started to think about persistence. You’ve heard about GLASS, you’ve got 64 bit hardware that’ll run VMWare and you’ve downloaded a copy of the appliance. You’ve clicked on the ‘Run Squeak’ menu item or you’ve executed the Smalltalk expression ‘GsLoginWindow open’ in a Squeak workspace, you’ve got Los Lobos’ ‘Beautiful Maria of My Soul’ queued up and you’re ready to rumba.

We’ll take a quick tour of the tools that will be available with version 2.2.4 to get your Cuban Motion in gear. Even if you’ve been using GLASS for awhile, it’s probably worth skimming this post since there are likely some new capabilities that you’ve been anxiously awaiting.

If you’re new to Squeak and Smalltalk, then you should check out the most excellent Squeak by Example or Steve Wessel’s excellent Squeak Tutorial. If you are new to Seaside, you should check out the Seaside Tutorial from Hasso-Plattner-Institut. If you’re going to do development in GemStone/S, then you should read the articles in GemStone 101.

Here’s a set of links for quick navigation to specific sections:

GemStone/S Login

The first thing you need to do is to establish a GemStone session. With the GemStone/S Login window, you specify the host name (or ip address) where the stone is running, the name of the stone, the host name (or ip address) where you want the gem to run, and the name or port number of netldi. For the Web Edition (i.e., free version), remote gems are not permitted, so the host name for the stone and the host name for the gem will always be the same.

When you press the “Login” button the Squeak image contacts the netldi which launches a gem on the stone’s host. Upon successful launching of the gem, the GemStone/S Transcript window is opened.

The gem creates a log file in the home directory (typically) of the user that owns the gem process (in the appliance that would be /home/glass). If the gem process ever exits unexpectedly, you can check in the log file for error messages.

The most common reason for a gem to exit unexpectedly is that it ran out of memory. If the gem runs out of memory, a fairly detailed report is dumped to the gem log. If there are no errors when the gem exits it will delete it’s log file. If you do run out of memory, you can bump up the size of the GEM_TEMPOBJ_CACHE_SIZE in the gem.conf (on the appliance it’s found in /opt/gemstone/product/seaside/data).

GemStone/S Transcript

The GemStone/S Transcript window is the control point for interacting with your GemStone session. For all of the GemStone tools, we are using Squeak as a windowing client. You want to remember that there is no interesting Smalltalk state being kept in your Squeak image, other than what is necessary for displaying the windows.

All of the GemStone tool windows have bland grey color and a “G/S[<GCI#>]” in the title bar to remind you that you’re in a GemStone window. You can open multiple sessions (gems) from a single Squeak image, so the GCI# helps you tell which session the window is operating against. Since the GCI library is not thread safe (i.e., only a single GCI call may be made at any one time), running multiple sessions from a single Squeak image is of limited utility.

The row of buttons across the top of window, give you quick access to the most commonly used windows and functions. Each window/function is covered in its own section:

The text pane in the GemStone/S Transcript window is actually a workspace and not a transcript. This window is slated to be reimplemented in OmniBrowser and we’ll fix the name then:). When you do an accept in this window, the text is stored in a class variable on the Squeak-side, so the text can be preserved across session boundaries. This is the one piece of state that would make it worth saving your Squeak image.

While we’re talking about image saving, it used to be that when you saved the Squeak image, all GemStone sessions were unilateraly destroyed rendering all of your GemStone windows inoperable. With the tools being released in conjunction with version 2.2.4, it is safe to save your Squeak image with GemStone windows open. When you restart a saved image, all of the open GemStone windows will be closed.

When you close the GemStone/S Transcript window (or press the Logout button), the GemStone session will be terminated (i.e., the gem process will exit) and all of the GemStone windows in the Squeak image associated with the session will be closed.

Keyboard Shortcuts

Back in December, while I was watching Avi write the Monticello tools in OmniBrowser, I saw him use some keyboard shortcuts that weren’t on the text pane menu. He was browsing classes and bringing up senders and implementors windows with a couple of key presses - things that I had been struggling to do without shortcuts.

I had been operating in a world where keyboard shortcuts show up on the menu! From Wikipedia on Keyboard shortcut (emphasis mine):

In most GUIs, a program’s keyboard shortcuts are discoverable by browsing the program’s menus - the shortcut is indicated in the menu choice.

Without going into the pros and cons, I decided to keep the menus short and follow the Squeak/OmniBrowser example. The following keyboard shortcuts are supported in all of the GemStone text panes whether they show up in a menu or not:

• CTL-b browse selected class
• CTL-N references for selected class
• CTL-n senders of selected selector
• CTL-m implementors of selected selector
• CTL-E methods containing selected string
• CTL-i inspect it
• CTL-p print it
• CTL-d do it

In workspaces, the evaluation context for CTL-i, CTL-p, and CTL-d, is not defined, which means that you can’t use ’self’ in workspace expressions. In code browsers the evaluation context is the selected class (’self’ is the class), in debuggers the evaluation context is the message receiver, and in inspectors the evaluation context is the inspected object (you have to select a field in the inspector to set the inspected object).

GemStone/S Workspace

The GemStone/S Workspace window is basically a clone of the GemStone/S Transcript with the exception that there is no Logout button and your session is not terminated when you close the window.

Monticello

I am going to write separate post to cover Monticello in GLASS. Suffice it to say that OmniBrowser-based Monticello tools closely parallel their Morphic-based cousins with the exception of a couple of functions that were not used a whole lot. If you find what you consider an important piece of functionality missing, by all means let me know via a comment on this blog, or join the news group described in the Bug Reports section.

For those of you who have been using the Morhpic-based Monticello tools. You will be happy to know that the arcane rigamarole that you used to have to go through after loading a package is no longer necessary. Changes to the code in a package will cause the package to be marked dirty, however, you may have to refresh a window to see the changes.

I have not implemented a window to window update mechanism, yet, so changes made in one window will not automatically show up in another window. Getting window updates to propogate is high on my priority list.

Browse…

When you press the Browse… button, you are prompted for a ‘Class name fragment’ (no need to worry about capitalization or including wildcard characters). If the fragment resolves to a single class, a System Browser is opened on that class. If the fragment resolves to multiple classes, you are given a list of classes from which to choose your desired class.

From the System Browser, you can navigate to the full complement of windows available in OB-Standard.

You might notice that in the class category pane, there is no ‘add category’ menu item available. GemStone manages it’s classes slightly differently than Squeak and the Class categories are derived from the classes. So when you create a class, you create the new category along with the class.

Find Method…

When you press the Find Method… button you are prompted for a ‘method fragment’ (no need to worry about capitalization or wildcard characters and multi-keyword selector fragments like ‘at:put’ are fine). If the fragment resolves to a single selector, then an Implementors window is opened, otherwise you are prompted to choose the selector of interest from a list.

The Implementors, Senders, and Variables windows are chasing browsers. The upper right pane contains a list of the messages sent by the method selected in the upper left pane. If you were to click lastValue in the upper right pane, you’d get a scroll bar below the selection lists, the left pane would be replaced by the messages pane and the upper tight pane would contain a list of implementors of #lastValue. You can scroll back to the left to see the original implementors left, or continue walking panes to the right. Chasing browsers are a little disconcerting to use, but once you realize how much the window clutter is reduced by using them, you will grow to like them.

Occasionally, when you accept a method in the Implementors, Senders, and Variables windows, the method text will appear to revert back to it’s original value. If you reselect the method, you will see that the source you accepted has actually been saved. There are a handful of these update bugs that I will be hunting down and eradicating as time goes by.

Test Runner

The Test Runner is one of the last remaing Morphic windows in the GemStone tool set. When you press the Test Runner button be prepared to wait a bit while a ton of data is transferred from the gem to the Squeak image. Updating after making a selection can take a bit as well, but I’d rather have a slow Test Runner window than none at all.

Not all of the functions in the Test Runner are functional for the GemStone tools, but ‘Run Selected’ works and if you get errors or test failures, clicking on the failure will bring up a GemStone debugger. Note also that ‘run test’ menu items can be found in the category, class, protocol and method panes of the code browsers.

Debugger

When you look at the debugger you immediately notice that the receiver and context inspector panes are missing! Until very recently it was difficult to get OmniBrowser to lay out a window configured like a traditional debugger. Lukas has been busy working on OB-Tools which implements a number of the Squeak windows using OmniBrowser and I think he may have a version of the debugger with integrated inspectors. I last merged with OB-Tools-lr.13 and Lukas is up to Ob-Tools-lr.19. Add one more item to my todo list.

To get around the lack of integrated inspectors, I added an ‘inspect context’ button that brings up an inspector that gives you combined view of the context and the receiver. The context inspector is slaved to the debugger, so that as you select different contexts the inspector updates its view to reflect the state of the selected context. I like the extra screen real estate afforded by the context inspector and dislike the fact that I am a couple of clicks farther away from seeing important information.

To facilitate Bug Reports, I’ve added a ‘copy stack’ menu that prints up a stack report including receiver and context info for each stack element.

Breakpoints

GemStone/S supports breakpoints, so it isn’t always necessary to edit a method to get the debugger to stop at an interesting point. The definition pane menu in the code browsers and debugger has a ’set breakpoint’ item that will set a breakpoint at the selected point in a method. For a method that has a breakpoint set, an additional menu item, ‘clear method breakpoints’, shows up and allows you to clear all of the breakpoints in the selected method. If a breakpoint is set on any method in the system, a ‘clear ALL breakpoints’ method shows up on all of the definition pane menus and will clear all the breakpoints in your session when selected.

A breakpoint browser is on the drawing board that will allow for the individual control of breakpoints, including disable/enable selected breakpoints.

Breakpoints are session-based and are associated with a particular instance of a method. If the method changes by either code changes on your part or the method is changed via an update via a commit or abort, then the previously set breakpoints aren’t active on the new instance of the method.

Inspector

Like the Implementors, Senders and Variables windows, the Inspector window is a chasing window. When you click on a field in the upper left inspector pane, the inspector for the object associated with the selection is shown in the upper right pane. The context for bottom workspace pane is derived from the selected object (not the selected field). Selecting a field in the upper right pane will bring up the scroll bar for the selection lists, sliding the upper right pane to the left, and showing the object inspector for the newly selected field in the upper right pane.

The example below happens to be a context inspector associated with the debugger window in the Debugger example.

Selecting the currentRequest field in the above example, shows an inspector on the instance of WARequest in the upper right pane. A field is selected for the instance of WAExpirySession in the upper left pane so a CTL-p in the bottom pane on escapeContinuation will be evaluated in the context of the WAExpirySession instance.

If you type an expression in the bottom pane and hit CTL-s, the value of the selected field will be set to the result of the expression (Liliana, you should be happy now:).

Abort/Commit Buttons

The Abort and Commit buttons in the GemStone/S Transcript window allow you to manually abort or commit your work.

When you press the Commit button it is the moral equivalent of saving your image in Squeak. The Abort button is the moral equivalent of quitting your Squeak image and restarting from your latest saved copy.

When you abort or commit, you are updated to the latest view of the repository. With an abort, all changes to persistent objects are discarded before the update. With a commit, your changes to persistent objects are incorporated into the latest view of the repository. For more information of transactions, take a look at the posts in GemStone 101.

Inadvertant use of the Abort button can be disastrous, so you are prompted before the abort is performed.

As discussed in GemStone 101: Transaction Conflicts, if during your interactive session you have executed code that has made changes to an object that was also changed by another session since your last abort, you will get a transaction conflict if you attempt to commit. If the commit fails due to transaction conflicts, you will see the following notification:

No matter which option you choose, you will ultimately have to abort your session before being allowed to commit again. Transaction conflicts do not happen very often during development, especially if you are working by yourself;).

In a group, a transaction conflict could occur if two developers made changes to the same class at the same time. For group development it makes a lot of sense to give each developer their own appliance (or userID) and use Monticello to share your code.

If you are doing development against a stone where the Seaside application is being hit by other users or processes (i.e., siege or WAPT) and you are actively updating shared data structures, you could very easily hit a transaction conflict.

Choosing to inspect transaction conflicts will bring up an inspector on the transacation conflicts Dictionary. Take a look at the comment in System class>>transactionConflicts for more details on the information contained in the Dictionary.

To create the above transaction conflict, I opened two GemStone/S Transcript windows, evaluated the expression ‘UserGlobals at:#Shared_TEST put: Object new’ in one workspace and pressed Commit button. This defined a new association in the UserGlobals SymbolDictionary. I pressed the Abort button in the other session’s transcript window and evaluated the same expression it it’s workspace and pressed the Commit button. Then in original transcript window (without pressing the Abort button first), I evaluated the expression again and pressed the Commit button, which gave me the transaction conflict. Knowing which object(s) are involved in the conflict should give you a pretty good clue as to where the problem might be located.

I am afraid that I may have broken the functionality of the Auto commit checkbox in the GemStone/S Transcript windows when I moved the browsers and Monticello to OmniBrowser. Given the frequency with which I press the Commit button, I will probably fix this feature in a future release, if noone else beats me to it:).

Hyper

If you just loaded some new code into the GemStone repository, or finished doing a bit of development, the easiest way to test out the new code is launch Hyper directly from your session, so that you can bring up the Debugger, if you encounter a problem.

The Hyper button prompts for a port number and launches a Hyper instance that is listening on the specified port number. You can then hit that port from your browser using a host name or ip address (i.e., http://172.16.172.134:9765/seaside or http://glass:9765/seaside or http://localhost:9765/seaside).

In older versions of GLASS, the Hyper-based web server required an exact match between the name of the host that you used to launch Hyper and the name of the host in the URL, which caused some unnecessary difficulties in connecting to the Hyper server.

When you launch Hyper, we block the UI process in Squeak. To interrupt the Hyper thread, you can hit Alt-.. Rember that for every HTTP request that comes in, the gem performs an abort, processes the request, then performs a commit. If you were trying to write code while this was going on, you’d easily lose chunks of work without warning.

If you want to do development and handle debuggable HTTP requests at the same time, the best solution is to fire off one Squeak image that is dedicated to running Hyper and use a second Squeak image to do development. When you’re done writing a batch of code, you can commit and the gem running Hyper will see the changes after the next request comes in.

If you hit a problem, you can use the debugger in the Hyper image to figure things out, just remember to commit your changes in the Hyper image before proceeding or restarting the Hyper process. Back in your development image, you’ll want to do an abort to pick up the changes that you made in the Hyper image. Breakpoints are session-specific, so you will need to set any breakpoints in your Hyper image, before launching Hyper.

If you find yourself trying to debug a problem that only shows up in FastCGI, you can arrange to start one or more Squeak images running FastCGI. For example, if you are using the appliance you could stop the threee gems that are servicing FastCGI requests and start three Squeak images to serve the FastCGI requests. Execute ‘FSSeasideHandler startUp: 9001′ in a GemStone workspace in the first image, ‘FSSeasideHandler startUp: 9002′ in the second, and ‘FSSeasideHandler startUp: 9003′ in the third. Alternatively, we’ve defined an Apache virtual host for each FastCGI port, so you can start just one Squeak image on FastCGI port 9001 (’FSSeasideHandler startUp: 9001′) and then hit http://glass:8081/seaside with your browser. Ports 8081, 8082, and 8083 bypass the Apache load balancer and route HTTP requests directly to the FastCGI ports 9001, 9002, or 9003 respectively.

Logout

When the Logout button is pressed, the session is terminated and all windows associated with that session are closed, including the GemStone/S Transcript window.

Bug Reports

With release of GemStone/S 64 version 2.2.4, the tools are finally of Beta quality. I’ve mentioned a couple of the known problems in this post, but my philosophy on bug reports is that you can’t have too many of them - don’t assume that we’ve seen the obvious problems, if you think it is a bug let us know.

To report bugs, sign up for our GLASS Beta mailing list and send a message to the list detailing the problem you’ve seen.

Even if you don’t have a bug report, I encourage to sign up for the GLASS mailing list. Seaside specific questions should still be directed to the Seaside mailing list. GemStone-specific questions can also be routed to the GemStone/S customer forum, most of our customers hang out there and are always willing to help folks that are new to GemStone/S.

David Buck has an announcement for the next Ottawa Smalltalk Users Group meeting where James Foster will be talking about GLASS. From David’s post:

While the Seaside framework elegantly addresses HTML generation and application flow-of-control issues, it still leaves challenges for the developer–including persistence, multi-user coordination, and scaling. With typical solutions (including object-relational mapping, external files, and multiple images) the “pure objects” experience of Smalltalk is compromised. In this presentation we will demonstrate GLASS (GemStone, Linux, Apache, Seaside, and Smalltalk), a stack (analogous to LAMP) that provides a robust environment for deploying sophisticated, dynamic web applications that can scale.

If you’re in the area, you should plan on heading over to the meeting.

Updated: [in a comment, Charles reminded me that James is hitting a trio of Smalltalk User Groups in Feburary, thanks Charles]:

• Toronto Smalltalk User Group - February 4, 2008
• Ottawa Carleton Smalltalk Users Group - February 5, 2008
• New York City Smalltalk Users Group - February 6, 2008

Ken Treis has started work on a MockGemStone package that provides implementations for a couple of the Rc classes discussed in GemStone 101:Transaction Conflicts. The package can be loaded into a Squeak image and the classes can be used to develop a Seaside application in Squeak that will run in both GemStone and Squeak environments.

To the extent that any tool set is ever actually done, I’m glad to say that the GemStone tools for GLASS are done. It has been a long haul, but I’m feeling real good about the state of the tools. I will write up a Terse Guide to the GLASS Tools in the next day or so to give you an idea of what to expect.

In order for you to actually get a look at the new tools you’ll have to wait for Version 2.2.4 of GemStone/S 64 or download a new version of the Seaside Appliance.

Version 2.2.4 is due to be released this Friday. We’re in the final stages of QA for this release and so far we are on track to make the date.

Once 2.2.4 goes out the door, we will build the new version of the appliance. It should be available next week. If you have been using an older version of the appliance, it is highly recommended that you download the new version. If you haven’t played with the appliance yet and have 64bit hardware drop us an email to get download instructions.

Between the holidays and time off to wrestle with a stubborn cold, December was a short, but busy month.

Version 2.2.4 of GemStone/S 64 is due to be released around the middle of January, so I’ve been scrambling to make changes to the base classes that are needed for OmniBrowser and Seaside.

We will also be cranking out a new rev of the GLASS appliance around the same time, so I’ve been working to get all of the image-based tests to pass and to clean up some of the Undeclared Symbols that have been floating around. Infrastructure improvements for the most part. With this rev of the GLASS appliance, we’ll still be in limited beta (i.e., if you have 64 bit hardware and want to take GLASS for a spin, drop us an email), but we are getting very close to making the beta available for download.

The big news is that we’ve finally got OmniBrowser-based Monticello tools, what a nice Xmas gift that was! We brought Avi down to Portland for a week and he nailed the Monticello tools. He did his development in Squeak, so we have made the results available to Damien Cassou (via the Package Universe) and he should be including the new tools in the beta Squeak development image. The OB-Monticello package is available in Colin’s OmniBrowser repository. I will be making a handful of changes in the next week or so as I get the new Monticello tools working in GemStone/S, but the plan is to have them ready for the next rev of the appliance. When that work is done, I’ll refresh the information in the Package Universe.

Finally, don’t forget that OOP 2008 in Munich (January 21-25, 2008) is coming up.

This is the second article in the GemStone 101 series. If you haven’t already done so, I recommend that you read GemStone 101: Transactions and Unlimited GemStone VMs in every Garage? ….and a Stone in every Pot before reading this post. It also wouldn’t hurt to take a gander at The Pillars of Concurrency for background on concurrency issues.

Transaction Conflicts

Today we’re going to delve a bit into the world of transaction conflicts. In GemStone/S a transaction conflict occurs when two or more sessions attempt to commit changes to the same object. A commit conflict is an error condition and in most cases you will end up aborting the transaction (for a discussion transaction conflicts in more detail read Chapter 6.1 in the GemStone/S Programming Guide). The best medicine for transaction conflicts is to avoid them.

[Updated: 3/17/2008] See GLASS 101: Simple Persistence for an alternative to conflict avoidance.

Object Locks

Avoiding transaction conflicts is really not that much different than dealing with concurrent access to a shared data structure in a single vm when multiple processes are running. In Squeak, you would simply protect shared data from concurrent access by using a Semaphore and a critical: block. In GemStone/S you can’t use Semaphores to protect persistent shared data, since each session is running in separate operating system processes (potentially on a different machine). You can, however, use object locks to control concurrent access to persistent shared data (See Chapter 6.3 in the GemStone/S Programming Guide for complete details on manipulating object locks).

Object locks are session based. Once a session obtains an object lock, other sessions will not be allowed to obtain a lock on the same object until the lock is released. For protecting access to shared data, you can arrange to release the lock on transaction boundaries (commit or abort), so that the access to the shared data is protected for the duration of the transaction.

If you are denied an object lock, you must wait for the lock to be released via polling or by using a variant of the object lock called an application lock. A request for an application lock will automatically block until the lock is available. Application locks are more convenient than standard object locks, but there are restrictions on the number of application locks that you can use.

Using a object lock on an object to protect access to shared data is a perfectly valid technique, but it should be used sparingly to avoid getting into potential deadlock situations and to avoid delays waiting for the lock to be released. Object locks are also relatively expensive to use, as they involve interaction with the stoned process.

Reduced Conflict Classes

As an alternative to using object locks to avoid transaction conflicts, GemStone/S provides a set of reduced conflict classes (See Chapter 6.4 in the GemStone/S Programming Guide for complete details on reduced conflict classes), that in many cases allow one to perform concurrent, conflict-free updates on the same object:

• RcCounter
• RcQueue
• RcIdentityBag
• RcKeyValueDictionary

In GemStone/S, a transaction conflict occurs when the same object is modified by two different sessions, even when the modifications are not logically inconsistent. For example, adding two different objects to an IdentityBag is not logically inconsistent, but will result in a transaction conflict if performed by two different sessions.

The reduced conflict classes were added to GemStone/S to allow logically consistent updates to be made to the same object by different sessions, while avoiding transaction conflicts.

Logically inconsistent operations on reduced conflict classes will still result in a failed transaction. For example, two sessions adding the same key to a RcKeyValueDictionary is logically inconsistent and will result in a failed transaction.

Reduced conflict classes should be used as your first line of defense in avoiding transaction conflicts.

RcCounter

A separate object is allocated for tracking the contributions from each session. The value of the RcCounter is calculated when you request its value by traversing over the contributions for each session.

RcQueue

For RcQueue, entries are timestamped and added to a separate collection for each producer session. A consumer session then ‘removes’ the elements from the queue keeping track of the elements removed without actually removing them from the producer session’s collection. You can avoid conflicts under the following conditions:

• Any number of sessions read objects in the queue at the same time.
• Any number of sessions add objects to the queue at the same time.
• One session removes an object from the queue while any number of sessions are adding objects.

RcIdentityBag

For RcIdentityBag, the contents are managed by keeping an ‘adds Bag’ and ‘removals Bag’ for each session. When you enumerate the contents, a cache of the bag (in transient session state) is created by adding and removing objects recorded for each session. You can avoid conflicts under the following conditions:

• Any number of sessions read objects in the bag at the same time.
• Any number of sessions add objects to the bag at the same time.
• One session removes an object from the bag while any number of sessions are adding objects.
• Any number of sessions remove objects from the bag at the same time, as long as no more than one of them tries to remove the last occurrence of an object.

RcKeyValueDictionary

For RcKeyValueDictionary, reduced conflict operation is achieved by recording adds and removes in a RedoLog. If a transaction conflict is encountered, objects within the dictionary are selectively aborted and the add and remove operations are replayed and a second commit is attempted. You can avoid conflicts under the following conditions:

• Any number of sessions read values in the dictionary at the same time.
• Any number of sessions add keys and values to the dictionary at the same time, unless a session tries to add a key that already exists.
• Any number of sessions remove keys from the dictionary at the same time, unless more than one session tries to remove the same key at the same time.
• Any number of users perform any combination of these operations.

Reduced Conflicts for Indexed Collections

In a future GemStone 101 article, I will go into a little more detail about indexed collections, suffice to say that if you need ordered access to, or need to perform queries on elements in a shared collection, you can use an RcIdentityBag with an RC equality index or two to reduce the likelihood of transaction conflicts.

Concurrency Control in Seaside

In GLASS we have modified the Seaside framework to use both object locks and reduced conflict classes to avoid transaction conflicts.

As I have written before, when a request comes into a GLASS vm, the framework does a beginTransaction before passing the HTTP request to the Seaside application code and the framework does a commitTransaction before passing the HTTP response to the user’s web browser.

Before we do the beginTransaction, we acquire a write lock on the WASession instance to ensure that no two vms are handling a request for the same WASession concurrently. The write lock is then released when the commitTransaction is performed. The object lock is the correct construct to use here, because the intent is to block other sessions from running concurrently. As an intended side effect, this also means that all session-specific data is protected from transaction conflicts.

Each WAApplication maintains a couple of handler dictionaries that are protected by a Semaphore in the standard Seaside code base. For GLASS, we use instances of the reduced conflict dictionary class RcKeyValueDictionary. In this case it isn’t necessary to completely block access to the dictionaries while they are updated. It is sufficient to ensure that the dictionaries are logically consistent at the end of the transaction - reduced conflict classes are the right answer in this case.

For the data structures (that aren’t already protected by the session lock) in your Seaside application, you can apply these tests to help decide what technique to use to avoid transaction conflicts:

• use write locks when concurrent updates to a data structure cannot be tolerated
• use reduced conflict classes when your usage pattern conforms to the restrictions imposed by the particular reduced conflict class

According to Andres Valloud, the Smalltalk Conference of Argentina appears to have been a success - there is already talk of a Smalltalks 2008. This is very good news for Smalltalk!

Judging from this picture by Edgar J. De Cleene (posted on Squeak-Dev mailing list) it was a packed house at James Foster’s “Getting Started with GLASS” workshop on day 3. Over 40 people attended the workshop, sharing 20 computers running the Squeak-based tools, and all doing work with GemStone/S running in a GLASS appliance on James’ 64bit laptop.

Back in October, I said that I was going heads down on the tools and I really wasn’t kidding. During November I didn’t blog very much, but I did make some maj