Running Aida/Web on GLASS

[1]

Janko Mivšek has provided instructions for installing AIDA/Web on GemStone/S 64.

——
[1] Photo by Satoru Kikuchi via Flickr (Creative Commons).

Installing GLASS on a KVM Virtual Machine

[1]

Miguel Cobá has provided instructions for installing GemStone/S 64 on a KVM Virtual Machine.

——
[1] Photo by wasabicube via Flickr (Creative Commons).

Why GemStone/S?

[1]

Isaiah Perumalla has a good post where he answers the question “Why GemStone/S?“:

…to eliminate the distinction between databases and programs, we need a technology that seamlessly integrates the capabilities of the programs and storage and multi user capabilities of databases…

——
[1] Photo by annnna via Flickr (Creative Commons).

GemStone 101: Managing Out of Memory Situations

[1]

You’ve been working in GLASS for a bit and wham! Every time you fire off this one expression, you run out of memory. Getting a

GemStone: Error         Fatal
VM temporary object memory is full
, old space overflow
Error Category: 231169 [GemStone]
  Number: 4067 Arg
  Count: 1
  Context : 20
  Arg 1:   20

isn’t fun. Before you head to Fry’s to buy more memory for your computer, there are a couple of things that you can do.

Batch Loading

Simple Commit

If you are doing a batch load, you can simply add a commit to your script inside a loop:

...do: [:i |
    System commitTransaction.
    ...]...

You can probably get away with doing a commit every couple of hundred iterations depending upon on how much data is being created.

Auto Commit

If there’s no convenient spot to insert a commit, you can build an AlmostOutOfOfMemory handler that will do a commit when you are in danger of running out of temporary object space. A caveat for using this script is that object structure that you’re building needs to be rooted in a persistent object.

| commitThreshold |
commitThreshold :=  75.
"Install AlmostOutOfMemory handler"
Exception
  category: GemStoneError
  number: (ErrorSymbols at: #rtErrSignalAlmostOutOfMemory)
  do: [:ex :cat :num :args |
      "Exception caught, do a commit."
      System commitTransaction 
          ifFalse: [ nil error: 'AutoCommit failed' ].
      "run a markSweep"
      System _vmMarkSweep.
      (System _tempObjSpacePercentUsedLastMark <
                commitThreshold)
          ifTrue: [
            "We dropped below the threshold,
             reenable the signal"
            System enableAlmostOutOfMemoryError ].
      "continue execution" ].
"Enable AlmostOutOfMemory signal"
System signalAlmostOutOfMemoryThreshold: commitThreshold.
[...put your code here...]
    ensure: [
        "disable AlmostOutOfMemory signal"
        System signalAlmostOutOfMemoryThreshold: -1].

Debugging Out of Memory Condition

If you’re not doing a bulk load, then you will need to debug the out of memory condition itself. Just today I was running unit tests and part way through the tests, the system ran out of temporary object space. I solved my problem and figured that it was a good idea to share the technique.

You start by enabling the out of memory signal (without defining a handler).

"enable AlmostOutOfMemory signal at 75% of
 temporary object space"
System signalAlmostOutOfMemoryThreshold: 75.

You can execute this statement in a workspace and it will be valid for your entire session. The debugger will come up when the available temporary object space crosses the 75% threshold allowing you to see what your application is doing when it runs out of memory.

If you need more information to help figure things out, the following script provides you with information about the objects that are filling up your temporary object space. See the method comment in System class>>_vmInstanceCounts: for additional options.

| ar totalSize instances sizes |
instances := IdentityDictionary new.
sizes := IdentityDictionary new.
totalSize := 0.
System _generationScavenge_vmMarkSweep.
ar := System _vmInstanceCounts: 3. "old space"
ar associationsDo: [:assoc |
  instances 
      at: assoc key name "class"
      put: (assoc value at: 1). "instance count"
 totalSize := totalSize + (assoc value at: 2).
  sizes 
       at: assoc key name "class"
      put: (assoc value at: 2) "total bytes" ].
^{ "label"
       '75% full'.
   "total size of objects in temporary object space"
       totalSize.
   "sorted list of classes and their total size in bytes"
       sizes associations sortWithBlock: [:a :b | 
           a value >= b value ].
   "sorted list of clases and their instance counts"
       instances associations sortWithBlock: [:a :b | 
           a value >= b value ]}

If you need more samples, you could wrap this script with an auto commit handler that would allow you to sample the population of temporary object space at various thresholds (say 25%, 50%, and 75%).

——
[1] Photo by Brent and MariLynn via Flickr (Creative Commons).

Scaffolding for GemStone

[1]

At Smalltalks 2008, James Foster gave a demo of “Scaffolding for GemStone”, a Seaside application written by Gerhard Obermann:

What Gerhard has done is used Magritte to model a domain, GemStone to save the objects, and Seaside to display them. In addition, Gerhard has created a web-based UI to interact with the Magritte data. Thus, after loading his package into GemStone, everything else is handled through a web browser. The tools build a basic “CRUD” user interface (that handles Create/Retrieve/Update/Delete for the domain data).

James created a 7-1/2 minute screencast showing off Scaffolding for GemStone. As James says:

Take a look at the demo and leave us your comments! Better yet, load the code into your GLASS environment and contribute to the open source project…

The code is available in the Scaffolding GemStone project on GemSource.

——
[1] Photo: Scaffolding, Uploaded by tilo driessen on 28 Jun 08, 6.50AM PST.

GemStone at Smalltalks 2008

[1]
Both Monty Williams and James Foster will be presenting later this week at Smalltalks 2008 – 2nd Argentinian Smalltalk Conference hosted by the Universidad Abierta Interamericana in Buenos Aires, Argentina.

Monty will be giving a keynote entitled “Smalltalk: Stories of Success” on Thursday at 10am:

The keynote includes stories about a number of successful Smalltalk projects – what problems they solved, how Smalltalk helped them succeed, and the changes they have had to deal with over the years.

James will be giving a tutorial on “GemStone Technology” on Thursday at 2:30pm:

Most Smalltalk implementations offer a strong platform for rich, client-side graphical user interface (GUI) applications. When it comes to persisting or sharing data, however, most systems end up using a relational database. While there are good object/relational (O/R) mapping tools available (such as GLORP), some people have found that the best solution is to avoid the O/R problem entirely. In this tutorial, we will look at GemStone/S, a command-line Smalltalk implementation (with no native GUI) that allows thousands of Smalltalk virtual machines to share the same object space (or image) with transactional persistence.

Then at 7:15pm James will be giving a talk entitled “GemStone Roadmap”:

GemStone introduced its Smalltalk application server and database technology at the first OOPSLA (held in Portland, Oregon, USA) in 1986. Since then, GemStone has become the leading Smalltalk object database and is used in a variety of large enterprises from finance to transportation. With Seaside support and the introduction last year of a free version of the product (with a 4 GB size limit), GemStone/S now is an attractive platform for startups looking for a “pure Smalltalk” solution for new web applications (no object/relational mapping is needed). In this presentation we will introduce GemStone/S and describe newly-released features in the products and give hints about what is being considered for upcoming releases.

You can download the entire schedule here(pdf).

——
[1] Photo: Universidad Abierta Interamericana, Uploaded by blmurch on 14 Dec 06, 2.43PM PST.

Pier Programming

[1]

This is the second post where I’m recording my experiences using Pier and GLASS for wife’s website. In the first post, I got GLASS and Pier installed on Slicehost using lighttpd as the web server.

Over the last week I didn’t have a lot of time to do very much with the Pier installation other than set things up so that Seaside would be safe on the open internet:

• Remove all of the default Seaside apps, except the object log.
• Add authorization to the object log. The object log is too useful to remove, but since you can get easy access to the inspector via the object log, it has to be sealed off.
  • Using the config editor for the object log, add WAAuthConfiguration as an Ancestor, then set the Login and Password.
• Set deploymentMode to true for pier, config and object log.
• Change all of the passwords (pier, config and object log).

My wife enjoyed herself writing articles and uploading pictures. We don’t have a domain name yet and we turn off lighttpd when we’re not actively using the site.

Today I finally found some time to play with Pier and take a chunk out of my todo list:

• Update to the latest GLASS package (GLASS.230-dkh.176 – still waiting for feedback from Dario, but I thought I’d go ahead and load it on my wife’s site)
• Fix a bug my wife found in the Pier-Blog package.
• Do a bit of repository cleanup.
• Convert the instances of MAMemoryFileModel to MAExternalFileModel,

Code Updates

The first thing I want to mention is that with a production site, it is a real good idea to have a sandbox where you can do development without worrying about impacting production users. Our web site won’t have a lot of traffic, but my wife puts in a fair amount of effort working on content and I don’t want to lose her work, so I take a couple of simple precautions:

1. Before making any significant changes, I make a backup of the extent file (/opt/gemstone/product/seaside/data/extent0.dbf). I shut down the stone before copying the file, but you can also use the runBackup script, which safely copies the extent file while the system is running.
2. Using scp, I copy the extent to my sandbox system (a 64 bit laptop running a copy of the appliance).
3. After shutting down the stone on my sandbox, I copy the extent file to the data directory and restart the stone. The copy is useful in case you do something regrettable.
4. After validating the upgrade/update steps using the sandbox, I perform them on the production system.

When all was said and done, I ended up committing a new version of the GLASS package (GLASS.230-dkh.177) and creating two web-site specific packages (Pier-Model-dkh.241 and Pier-Seaside) that I saved in a directory-based repository.

Repository Cleanup

Looking at the object log, I had seen that the repository had grown from 100M to 160M since we’d started, and I was interested in seeing what I could do to reduce the amount of space used:

• There were a couple of continuations stored in the object log that I could get rid of now that the bug that my wife ran into was fixed.
• I knew that the all of the Monticello packages are saved in the repository in the Monticello cache, so I was interested to see what could be gained by flushing the cache.
• While working out the details of converting from MAMemoryFileModel to MAExternalFileModel I noticed that we had some 500 entries in the PRHistoryPersistency. Each one of history entries hangs onto a WASession along with other things, so I figured that cutting the size of the history list to 250 (from 1024) wouldn’t hurt.

In the sandbox, I gained about 20M by doing the above cleanup steps, but some of the Monticello operations took noticeably longer (the cache had to be refilled), so I don’t think that flushing the Monticello cache is worth it.

Convert from MAMemoryFileModel to MAExternalFileModel

GemStone/S extent files get moved around a lot between the time that the Seaside extent is created on one of our build machines and the time an appliance is fired up or GLASS is installed on a Linux or MAC machine. This movement makes it extremely impractical to do anything that depends upon a fixed directory path and lead to my decision to make MAMemoryFileModel the default mechanism used by Magritte and Pier to manage it’s files for GLASS.

Once a system is installed in a production environment it is desirable to start using MAExternalFileModel, because you can arrange for your web server to serve up the files.

In our case, my wife had already uploaded a number of images as instances of MAMemoryFileModel, so I needed to convert the existing instances as well as change Pier to start using MAExternalFileModel.

The first thing to do is to edit PRFile class>>descriptionFile to replace MAMemoryFileModel with MAExternalFileModel.

Then you need to decide where you want the files to be stored. If you are using the appliance, then Apache serves files out of ‘/opt/gemstone/apache/htdocs’. If you are using lighttpd as set up by Miguel Cobá, then files are served out of ‘/var/www’.

The script below, illustrates the choices made for an appliance. The rest of the script grabs all PRFile instances and swaps in an equivalent instance of MAExternalFileModel. The final statement in the script flushes the MADescriptBuilder cache, otherwise you’ll continue using MAMemoryFileModel:

|  pf fm xfm |
MAExternalFileModel baseDirectory:
    (ServerFileDirectory on: '/opt/gemstone/apache/htdocs/files').
MAExternalFileModel baseUrl: '/files'.
System commitTransaction ifFalse: [ nil error: 'Commit failed' ].
(PRFile allInstances select: [:each | 
    each file isKindOf: MAMemoryFileModel ]) do: [:pf |
        fm := pf file.
        xfm := MAExternalFileModel new
            filename: fm filename;
            mimetype: fm mimetype;
            contents: fm contents asByteArray;
            yourself.
        pf file: xfm].
MADescriptionBuilder default flush.

This pretty much takes care of the major items on the todo list for my wife’s website.

I am playing with some code that generates real clean URLs for Pier (unfortunately, there are some gotchas for the moment). My wife has seen some java script effects on other sites that she’d like to be able to do on her site so I’ll there’s some java script in my future. When my wife is happy with the way things look, we’ll get that domain name we’ve had our eyes on:) and set up the google analytics.

I have to say that I’m real pleased with the Pier is working out for both me and my wife.

——
[1] Photo: 2007-08-12, Uploaded by sayzey on August 16, 2007.