GLASS Beta Update: Working with SOAP (Preview)

[1]

If you find yourself wishing that you could work with SOAP and GLASS, you don’t need to head out to the shed and drag out the old bucket and squeegee. All you need to do is load up the latest preview GLASS version and you’ll be ready to go!

HTTP Server and Client

Before getting started with SOAP, we need to talk a little bit about Hyper. SOAP needs an HTTP client to access services and an HTTP server to provide services. While there are a number of good Smalltalk-based HTTP server implementations floating around out there, there aren’t very many good HTTP Clients!

Fortunately,  Bruce Badger has been continuously updating his Hyper HTTP server and last fall he released a version of Hyper with a very nice HTTP Client implementation. In November, I used the Hyper HTTP Client in a port of Masashi Umezawa’s SoapCore and created a GLASS.231 branch. I needed to create a branch, because the new Hyper was incompatible with the version of Swazoo that I was using for Seaside.

Recently I’ve received a number of requests for SOAP support in GLASS, so I bit the bullet last week and tackled the Seaside/Swazoo/Hyper issue. After a couple of days working through various solutions, I decided to ditch Swazoo and write a simple service class for gluing together Hyper and Seaside. See the section on Using Hyper for details.

AIDA

To be fair, the Swazoo code used in GLASS was several years old (at least). Janko Mivšek has a newer version of Swazoo (Swazoo 2) ported to GLASS, but for SOAP support I need an HTTP Client and the latest Hyper is the ticket. To use Aida with GLASS going forward, you should probably unload the Hyper package, before following Janko’s instructions for running Aida/Web on GLASS.

SOAP

Back in November, I was able to find 3 SOAP servers in the wild that were still up and running (4s4c, Microsoft Soap Interop Server and EasySoap++ – as of this writing 4s4c appears to be MIA). The servers were built based on the ROUND 2 SOAP Interoperability Tests Specification a (proposed) SOAP service interface that can be implemented by SOAP implementations as part of an interoperability testing process. This page is the best resource for information about the Interoperability Tests and this page has the results of the Interoperability Tests as of November 2001. The tests were arranged by the SOAP Builders Yahoo group and this looks like the message that kicked things off.

I was happy to find a set of servers out there that were still alive, since the servers for the original interop tests built into SoapCore were no longer available! I then built a client and server on top of SoapCore (SoapInteropClient and SoapInteropServiceImpl) and a set of TestCases (SoapInteropTestCase) so that I could test the GLASS SoapCore implementation against the 3 external services and an internal service. By default the test cases for the external services are disabled (see SoapInteropTestCase class>>skipInteropTesting: for information about enabling the tests against the external services).

If you find that you need to use SOAP with a project start by looking at the interop examples. Also read the sections GemStone Processes and GCI and SOAP/HTTP Server Development Tips for information on doing SOAP development in GLASS.

SOAP Server Transactions

I have not done any work to add transparent persistence to the SOAP server, but I would imagine that the approach used by FastCGISeaside>>answerResponderRoleCheckingLock: and SeasideHTTPService>>answerToCheckingLock: will work for SOAP. The same rules apply:

• Perform an abort before handling the SOAP request and perform a commit before returning the SOAP response. If no persistent objects were modified while serving the request (a read only request for example), the final commit is no more expensive than an abort.
• One concurrent transaction per gem. Add more gems for handling concurrent requests (presumably Apache or lighttpd can be used to round robin the SOAP requests).
• On a denied lock or transaction conflicts abort and retry the request a short while later.

If anyone is interested in adding transactions to a SOAP server let me know and I’ll whip out a prototype. Right now I assume that the primary interest in SOAP is as a client.

WSDL

Paul DeBruicker attended the GLASS Workshop in Santa Fe earlier this month and recently Paul announced (beta mailing list subscription required) his desire to port the Spray client from Dolphin to GemStone/S. Spray is a Dolphin-based package written by Steve Waring that adds WSDL support to SoapCore.

Nice timing.

Hernan Wilkinson has offered to work on the project along with Paul and I’m sure if anyone else interested in SOAP/WSDL support for GLASS, that your help would be appreciated. When it comes to web servers, I’m sure that the more players involved in interoperability testing the better the final product. Join and send mail to the beta mailing list if you would like to participate.

Using Hyper: SeasideSite replaced by SeasideHTTPService

“Ditching Swazoo” is not quite as bad as it sounds. Anyone using FastCGI will not be affected and the new version of Hyper looks to be an improved version.

If you use Hyper in production, then I would recommend that you move forward cautiously. I haven’t put the new Hyper under stress for this preview release. I intend to do so before making a beta release. Hopefully the next beta release (non-preview) will be the last beta release, so I plan to subject the system to some heavy load for both FastCGI and Hyper. As it currently stands, Hyper passes the unit tests and I’ve done limited manual testing.

If you are using $GEMSTONE/seaside/bin/startSeaside_Hyper, then you need to edit the file and replace the chunk of code that looks like this:

server := SwazooServer singleton.
server initialize.

site := (SeasideSite new)
    name: 'Seaside';
    initialize;
    host: GsSocket getLocalHostName
        ip: (GsSocket getHostAddressByName: GsSocket getLocalHostName)
        port: $1;
    initializeResourceFromUriPattern: '/', 'seaside'.

server addSite: site.

GsFile gciLogServer: 'Hyper Server started on port ', $1 printString.
SwazooServer startSite: 'Seaside'.
"does not return, unless there's an error"

with a chunk of code that looks like this:

server := SeasideHTTPService onPort: $1 acceptInForeground: true multiThreading: true.
GsFile gciLogServer: 'Hyper Server started on port ', $1 printString.
server start.
"does not return, unless there's an error"

If you have been using SeasideSite launchSeaside: 9765 to run Hyper using a GemTools Client, then you should use SeasideHTTPService startServiceOnPort: 9765 instead.

Finally don’t skip the sections GemStone Processes and GCI and SOAP/HTTP Server Development Tips, since they contain valuable information on doing Hyper development in a GemTools Client.

GemStone Processes and GCI

When you do a login from a GemTools Client, a separate Gem process is started on the stone’s machine. The client Smalltalk image and the Gem process communicate via GCI using a relatively simple protocol:

• Commands are initiated by the client (most often sending a message to to an object or executing code in a string). When a GCI command is executed in the Gem a new GsProcess is created and the command is run within the context of that new process.
• While the command is executing in the Gem, the client is blocked waiting for a response (there is a non-blocking mode for GCI which we use for in the GemTools Client, however, the operation is still blocking as far as the end user is concerned, because the GCI API is not thread safe).
• If you hit ALT-. in your Squeak/Pharo image, we catch the client-side interrupt and use the GCI to send a Hard Break to the GsProcess running in the Gem. This results in a GemStone debugger being opened. If you close the debugger the GsProcess is terminated. If you Proceed, the GCI API is used to resume execution of the interrupted GsProcess.
• Unhandled Errors that occur during execution on the Gem are passed over the GCI to the client where we arrange to bring up a GemStone debugger. Note: if you use the [unset|set] UsePopUpOnDebug command on the Admin… menu you can arrange to open a Squeak/Pharo debugger on the client-side Process.
• A command finishes when the GsProcess created to run the command ‘falls off the end’ (like a doIt) or there is an explicit return in the executed code. In either case the result of the command is returned across the wire to the client.
• Execution of code in the Gem blocks until another GCI command is received.

The Gem blocks when a command finishes execution. If you execute a command like:

[[ true ] whileTrue: [ (Delay forSeconds: 10) wait ]] fork.

The command will return immediately and the Gem blocks until you execute another GCI command.

For normal development activities, this doesn’t make too much difference, but if you are trying to debug a web server or a SOAP server, then you need be aware that unless you have explicitly given the Gem control (via a GCI call), your Gem threads will be blocked.

SOAP/HTTP Server Development Tips

A lot of issues can be fleshed out using unit tests, but occasionally you hit an issue that only appears when the system is running as a server. For Seaside applications we’ve arranged for continuations to be used for the remote debugging of application errors.

I have not added continuation-based debugging to the SOAP server (could be done if there is interest), so in order to debug a SOAP server, you’ll need to run your development vm as a server. You need to do the same thing if you want to debug internal server errors for Hyper or FastCGI.

Given that you understand that your Gem threads are blocked:) while your GemTools Client is active and given that you’ve opened a Process Browser you are ready to do server-side development. At the end of this section, I provide instructions for running SOAP/Hyper/FastCGI servers within your development vm.

For the purposes of discussion, I’ll assume that you’ve started a Hyper server and that you’ve navigated to the WAErrorTest page (click on the image for full size view):

Click on the Raise error link and click on the Debug link and the debugger will pop up in the GemTools Client:

The third process involved is the main hyper thread that is listening on port 9765:

So we’ve got three processes of interest: the main accept connection loop for Hyper, a thread forked off by Hyper to service the browser connection, and the a thread forked off by Seaside to service the Seaside request. We’ve got a debugger open on the the third thread and since we signaled an Error, the exception isn’t resumable. If we close the debugger control will be maintained by the client and the Gem execution is blocked.

Once the debugger is closed you can do a couple of things:

• Select each of the other two process and use the terminate menu item to terminate the process. If you go this route, you’ll (obviously) need to restart the Hyper server to serve HTTP requests.
• Select the process that is waiting on a Semaphore and terminate it. If you go this route, you simply need to the use the yield forever menu item on the Process Browser to return contol to the Gem and allow the processes to run freely.
• You should also be able to simply use yield forever without terminating any process – if you look closely you’ll see that the process in the debugger has an #ensure: block that should #signal the semapore that the other process is waiting on. Unfortunately, that doesn’t correctly at the moment. The fact that we’re bringing up a debugger with a hard break, is a special case and I’ll be looking at this bug more closely.

If you hit Alt-. while the server has control, control will be returned to the client, however, you can’t determine which process will be interrupted – the process that is running when the vm detects the interrupt will have the debugger brought up on it.

For all of the server variants, internal server errors are logged in the Object Log – commits are needed in the Soap server to make the log entries persistent. Take a look at the object log after running the Soap unit tests, for examples.

The following three sections provide specific information for debugging server code.

Soap Server

The following expression starts an HTTPServer listening on port 8823 that is ready to serve SOAP requests from the ROUND 2 SOAP Interoperability Tests Specification:

SoapInteropServiceImpl registerAllServices.
(Delay forMilliseconds: 100) wait.

The Soap service is started in the background. so the Delay is necessary to allow the server thread to make enough progress to be ready to service client requests. This setup is aimed at convenience for running tests. For a production server, you should arrange for the HTTPServer to be started running in the foreground (see SoapHTTPService class>>named:onPort:).

If you are trying to debug a Soap server error, here are a couple of useful methods to set breakpoints in:

• SoapHttpConnector>>send:to:with: in the SoapHttpGenericError handler.
• SoapEnvelopeBuilder>>buildSoapEnvelopeFromXmlString: in the Error handler.
• SoapServiceHandler>>dispatch:with: in the Error handler.
• HTTPConnection>>internalErrorHandler.

Hyper Server

The following expression starts an HTTPServer listening for HTTP requests on port 9765:

SeasideHTTPService startServiceOnPort: 9765

If you are trying to debug a Hyper server error, here are a couple of useful methods to set breakpoints in:

• SeasideHTTPService>>internalServerMalfunction:
• HTTPConnection>>internalErrorHandler.

FastCGI Server

For FastCGI you need to edit your Apache (or lighttpd) config to route all requests to a single server. You also need to stop any FastCGI gems that may be running (you can use the Stop Seaside gems command on the Admin… menu). Then run the following expression to start a FastCGI server listening on port 9001 in your development vm:

FSSeasideHandler startUp: 9001.

If you are trying to debug a FastCGI server error, here are a couple of useful methods to set breakpoints in:

• FsSeasideHandler>>internalServerMalfunction:
• FSConnection>>handleError:
• FSConnection>>handleFSError:
  

GemStone-dkh.466

A number of commands were added to the GemTools Launcher.

• [unset|set] UsePopUpOnDebug command added to Admin… menu. If you consistently get an out-of-memory error it may be caused by a recursive error. Setting UsePopUpOnDebug causes a pre-debugger window popup to appear. If the pop-up appears instread of an out-of-memory notifier, select the Debug Squeak stack option and send the Squeak stack as a Bug Report to the beta mailing list for analysis. If you choose Debug GemStone stack the normal GemStone debugger will be opened (although in an out-of-memory situation, it is likely that there is an error opening the GemStone debugger).
• System Browser command added to Tools… menu.
• Mark For Collection command added to Admin… menu.
• Browser Preferences submenu to Admin… menu which allows you to set preferences directly on server instead of inheriting from client (which might require an logout/login sequence).
• Override FFI>>finalize with a bugfix that eliminates that annoying finalize error that occasionally pops up.
• JadeServer class moved to server-side for faster logins. A version of JadServer still exists on the GemTools Client for backwards compatibility with older server versions.

Take a look at the GemTools change log for a complete history of changes since GemStone-dkh.449.

GLASS.230-dkh.229

In addition to the Hyper overhaul and addition of SOAP support:

• finish fixing Bug39828 – fastcgi and uploadfile problems (thanks Dario)
• suspend, terminate, yield 5, and yield forever commands were added to the Process Browser.
• Several packages have been  removed from the GLASS package during the preview process. Normally these orphaned packages remain in your repository unless you explicitly unload them. However, with GLASS.230-dkh.220 I decided that the following 5 packages needed to be explicitly removed (via preDoIt): SwazooSeaside, Swazoo, SqueakSource, MonticelloConfigurations, TinyWiki. If these packages are not explicitly unloaded, the unit tests will not pass.
• JadeServer class moved to server-side for faster logins.
• The package GemStone-Scaffolding was added to GLASS. See James Foster’s Scaffolding for GemStone screencast for more information.
• The #defaultAction for Warning has been changed to bring up a confirmation window which will permit you to continue or bring up a debugger. This means that you’ll start seeing warning messages during Monticello operations that were silent before.

Take a look at the GLASS change log for a complete history of changes since GLASS.230-dkh.216.

Update Process

To update an existing installation to GemStone-dkh.466 and GLASS.230-dkh.229, follow these instructions. While it is possible to use the Monticello Browser for upating, I am now recommending that you use the Update Client and Update GLASS menu items on the Update… menu to avoid issues that may be introduced by UI changes.

When you update to GLASS.230-dkh.229 a Warning dialog will pop up (due to the changes in the #defaultAction for Warning) – proceed through warning.

—–

[1] Photo by sea turtle (Creative Commons).