Using nXML mode in Emacs

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

Resources

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

The Request Processor

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

By Pete Su

OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff.

This document is a brief introduction to the OpenACS 5.2.3rc1 Request Processor; more details can be found in the OpenACS 4 Request Processor Design. Here we cover the high level concepts behind the system, and implications and usage for the application developer.

The 5.2.3rc1 Request Processor is a global filter and set of Tcl procs that respond to every incoming URL reaching the server. The following diagram summarizes the stages of the request processor assuming a URL request like http://someserver.com/notes/somepage.adp.

Stage 1: Search Site Map

The first thing the RP does is to map the given URL to the appropriate physical directory in the filesystem, from which to serve content. We do this by searching the site map data model (touched on in the Packages, and further discussed in Writing OpenACS Application Pages). This data model maps URLs to objects representing content, and these objects are typically package instances.

After looking up the appropriate object, the RP stores the URL, the ID of the object it found, and the package and package instance the object belongs to into the environment of the connection. This environment can be queried using the ad_conn procedure, which is described in detail in OpenACS 4 Request Processor Design. The page development tutorial shows you how to use this interface to make your pages aware of which instance was requested.

Stage 2: Authentication

Next, the Request Processor examines the request for session information. Session information is generally sent from the client (the user's browser) to the server via cookies. The security/session handler is described in detail in its own document. It examines the client request and either extracts or sets up new session tokens for the user.

Stage 3: Authorization

Next, the Request Processor checks if the user has appropriate access privileges to the requested part of the site. In OpenACS 5.2.3rc1, access control is dictated by the permissions system. In this case, the RP checks if the user has "read" priviledges on the object in the site map specified by the URL. This object is typically a package instance, but it could easily be something more granular, such as whehter the user can view a particular piece of content within a package instance. This automatic check makes it easy to set up sites with areas that are only accessible to specific groups of users.

Stage 4: URL Processing, File Search

Finally, the Request Processor finds the file we intend to serve, searching the filesystem to locate the actual file that corresponds to an abstract URL. It searches for files with predefined "magic" extensions, i.e. files that end with: .html, .tcl and .adp.

If the RP can't find any matching files with the expected extensions, it will look for virtual-url-handler files, or .vuh files. A .vuh file will be executed as if it were a Tcl file, but with the tail end of the URL removed. This allows the code in the .vuh file to act like a registered procedure for an entire subtree of the URL namespace. Thus a .vuh file can be thought of as a replacement for filters and registered procs, except that they integrate cleanly and correctly with the RP's URL mapping mechanisms. The details of how to use these files are described in OpenACS 4 Request Processor Design.

Once the appropriate file is found, it is either served directly if it's static content, or sent to the template system or the standard Tcl interpreter if it's a dynamic page.

Once the flow of control reaches a dynamic page, the Request Processor has populated the environment of the request with several pieces of useful information. The RP's environment is accessible through the ad_conn interface, and the following calls should be useful to you when developing dynamic pages:

[ad_conn user_id]

The ID of the user associated with this request. By convention this is zero if there is no user.

[ad_conn session_id]

The ID of the session associated with this request.

[ad_conn url]

The URL associated with the request.

[ad_conn urlv]

The URL associated with the request, represented as a list instead of a single string.

[ad_conn file]

The actual local filesystem path of the file that is being served.

[ad_conn object_url]

If the URL refers to a site map object, this is the URL to the root of the tree where the object is mounted.

[ad_conn package_url]

If the URL refers to a package instance, this is the URL to the root of the tree where the package is mounted.

[ad_conn extra_url]

If we found the URL in the site map, this is the tail of the URL following the part that matched a site map entry.

[ad_conn object_id]

If the URL refers to a site map object, this is the ID of that object.

[ad_conn package_id]

If the URL refers to a package instance, this is the ID of that package instance.

[ad_conn package_key]

If the URL refers to a package instance, this is the unique key name of the package.

[ad_conn path_info]

In a .vuh file, path_info is the trailing part of the URL not matched by the .vuh file.

Install tclwebtest.

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

cd /tmp
cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/tclwebtest co tclwebtest
#wget http://umn.dl.sourceforge.net/sourceforge/tclwebtest/tclwebtest-1.0.tar.gz
#tar xvzf tclwebtest-1-0.tar.gz
mv tclwebtest-0.3 /usr/local/
ln -s /usr/local/tclwebtest-0.3 /usr/local/tclwebtest
ln -s /usr/local/tclwebtest/tclwebtest /usr/local/bin

Installing SSL Support for an OpenACS service

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

How to package and release an OpenACS Package

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

Release Version Numbering

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

5.0.0
5.0.0rc2
5.0.0rc1
5.0.0b4
5.0.0b1
5.0.0a4
5.0.0a3
5.0.0a1
5.0.0d1
4.6.3

> cvs log readme.txt
RCS file: /usr/local/cvsroot/acs/readme.txt,v
Working file: readme.txt
head: 3.1
branch:
locks: strict
access list:
symbolic names:
	acs-4-0: 3.1.0.8
	acs-3-2-2-R20000412: 3.1
	acs-3-2-1-R20000327: 3.1
	acs-3-2-0-R20000317: 3.1
	acs-3-2-beta: 3.1
	acs-3-2: 3.1.0.4
	acs-3-1-5-R20000304: 1.7.2.2
	acs-3-1-4-R20000228: 1.7.2.2
	acs-3-1-3-R20000220: 1.7.2.2
	acs-3-1-2-R20000213: 1.7.2.1
	acs-3-1-1-R20000205: 1.7.2.1
	acs-3-1-0-R20000204: 1.7
	acs-3-1-beta: 1.7
	acs-3-1-alpha: 1.7
	acs-3-1: 1.7.0.2
	v24: 1.5
	v23: 1.4
	start: 1.1.1.1
	arsdigita: 1.1.1
keyword substitution: kv
total revisions: 13;	selected revisions: 13
description:
...

    <version ...>
        <provides .../>
        <requires .../>
        <maturity>1</maturity>
        <callbacks>
            ...
    

Set up Log Analysis Reports

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

Internationalization

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

Webtest

Created by Anett Szabo, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

API testing is only part of testing your package - it doesn't test the code in our adp/tcl pairs. For this, we can use TCLWebtest (see sourceforge).

TclWebtest is primarily for testing user interface and acceptance testing. It is a tool to write automated tests for web applications. It provides a simple API for issuing http requests, dealing with the result and assume specific response values, while taking care of the details such as redirects and cookies.
It has some basic html parsing functionality, to provide access to elements of the result html page that are needed for testing (mainly links and forms).

• TCLWebtest provides a library of functions (see command reference) that make it easy to call a page through HTTP, examine the results, and drive forms. TCLwebtest's functions overlap slightly with acs-automated-testing; see the example provided for one approach on integrating them.

• TCLWebtest tries to minimize the effort to write tests by implicitely assuming specific conditions whenever it makes sense. For example it always expects the server to return http codes other than 404 or 500, unless otherwise specified.

• The assertion procedures are targeted at test writers who want to make sure the behaviour of their web applications stays the same, without caring for style or minor wording changes. In the example below, it is just assumed that there is a link with the text "login" on the first page, that clicking on it results in a page with at least one form with at least two text-entry fields on it, and that submitting the form with the specified values results in a page that contains the "logged in" text.

• TCLWebtest should be suitable for testing larger chains of user interaction on a web application, for example a full ecommerce ordering session. tclwebtest could visit an ecommerce site as anonymous user, add some products to its shopping cart, check out the cart, register itself as user and enter a test address etc. The test script could also include the administration part of the interaction, by explicitely logging in as site admin, reviewing and processing the order, nuking the test user etc.

• TCLWebtest must be installed for to work. Since automated testing uses it, it should be part of every OpenACS installation. Note that TCLwebtest is installed automatically by Malte's install script.

Hint:

In order to simplify the generation of tclwebtest scripts the  Webtest-Recorder extension (TwtR) for Firefox is available see http://www.km.co.at/km/twtr This module is a plugin for Firefox. It is used to generate/edit a tclwebtest script which can be used later for regression testing without the need of a browser. There is a certain overlap of the application range between selenium and TwtR. This plugin was developed by Åsmund Realfsen for regression/load testing of the assessment module.

A typical script for tclwebtest looks like this:

set SERVER "testserver"
do_request "http://$SERVER/sometesturl/"
assert text "some text"

link follow "login"

field fill "testuser"
field fill "testpassword"
form submit

assert text "you are logged in as testuser"
    

 http://cvs.openacs.org/cvs/openacs-4/etc/install/tcl/twt-procs.tcl?rev=1.18

Command Reference:

• http://tclwebtest.sourceforge.net/doc/api_public.html
  

Here are some guidelines on how to write automated tests with TCLWebtest. It is a joy to work with automated testing once you get the hang of it. We will use the "myfirstpackage" as an example.

Create the directory that will contain the test script and edit the script file. The directory location and file name are standards which are recognized by the automated testing package:

[$OPENACS_SERVICE_NAME www]$ mkdir /var/lib/aolserver/$OPENACS_SERVICE_NAME/packages/myfirstpackage/tcl/test
[$OPENACS_SERVICE_NAME www]$ cd /var/lib/aolserver/$OPENACS_SERVICE_NAME/packages/myfirstpackage/tcl/test

[$OPENACS_SERVICE_NAME test]$ emacs myfirstpackages-procs.tcl

Write the tests. This is obviously the big step :) The script should first call ad_library like any normal -procs.tcl file:

ad_library {
    ...
}

To create a test case you call aa_register_case test_case_name.. Once you've created the test case you start writing the needed logic. We'll use the tutorial package, "myfirstpackage," as an example. Let's say you just wrote an API for adding and deleting notes in the notes packages and wanted to test that. You'd probably want to write a test that first creates a note, then verifies that it was inserted, then perhaps deletes it again, and finally verifies that it is gone.

Naturally this means you'll be adding a lot of bogus data to the database, which you're not really interested in having there. To avoid this I usually do two things. I always put all my test code inside a call to aa_run_with_teardown which basically means that all the inserts, deletes, and updates will be rolled back once the test has been executed. A very useful feature. Instead of inserting bogus data like: set name "Simon", I tend to generate a random script in order avoid inserting a value that's already in the database:

set name [ad_generate_random_string]

Here's how the test case looks so far:

aa_register_case mfp_basic_test {
    My test
} {
    aa_run_with_teardown  -rollback  -test_code  {

       }
}

Now look at the actual test code. That's the code that goes inside -test_code {}. We want to implement test case API-001, "Given an object id from API-001, invoke mfp::note::get. Proc should return the specific word in the title."

      set name [ad_generate_random_string]
      set new_id [mfp::note::add -title $name]
      aa_true "Note add succeeded" [exists_and_not_null new_id]

To test our simple case, we must load the test file into the system (just as with the /tcl file in the basic tutorial, since the file didn't exist when the system started, the system doesn't know about it.) To make this file take effect, go to the APM and choose "Reload changed" for "MyFirstPackage". Since we'll be changing it frequently, select "watch this file" on the next page. This will cause the system to check this file every time any page is requested, which is bad for production systems but convenient for developing. We can also add some aa_register_case flags to make it easier to run the test. The -procs flag, which indicates which procs are tested by this test case, makes it easier to find procs in your package that aren't tested at all. The -cats flag, setting categories, makes it easier to control which tests to run. The smoke test setting means that this is a basic test case that can and should be run any time you are doing any test. (a definition of "smoke test")

Once the file is loaded, go to ACS Automated Testing and click on myfirstpackage. You should see your test case. Run it and examine the results.

Example

Now we can add the rest of the API tests, including a test with deliberately bad data. The complete test looks like:

ad_library {
    Test cases for my first package.
}

    
aa_register_case  -cats {smoke api}  -procs {mfp::note::add mfp::note::get mfp::note::delete}  mfp_basic_test  {
A simple test that adds, retrieves, and deletes a record.
} {
aa_run_with_teardown  -rollback  -test_code {
set name [ad_generate_random_string]
set new_id [mfp::note::add -title $name]
aa_true "Note add succeeded" [exists_and_not_null new_id]
# Now check that the item exists
mfp::note::get -item_id $new_id -array note_array
aa_true "Note contains correct title" [string equal $note_array(title) $name]
# Now check, if titel got the value of name


   mfp::note::delete -item_id $new_id
          
                set get_again [catch {mfp::note::get -item_id $new_id -array note_array}]
                aa_false "After deleting a note, retrieving it fails" [expr $get_again == 0]
            }
    }
            
  

aa_register_case  -cats {api}  -procs {mfp::note::add mfp::note::get mfp::note::delete}  mfp_bad_data_test  {
        A simple test that adds, retrieves, and deletes a record, using some tricky data.
    } {
        aa_run_with_teardown  -rollback  -test_code  {
                set name {-Bad [BAD] \077 { $Bad}}    

                #Now name becomes this very unusual value: -Bad [BAD] \077 { $Bad}
                append name [ad_generate_random_string]
                set new_id [mfp::note::add -title $name]     

                #Now new_id becomes the value of the solution of proceduer add with starting argument $name as -title
                aa_true "Note add succeeded" [exists_and_not_null new_id]
                #Now test that new_id exists
                mfp::note::get -item_id $new_id -array note_array
                aa_true "Note contains correct title" [string equal $note_array(title) $name]
                aa_log "Title is $name"
                mfp::note::delete -item_id $new_id

                set get_again [catch {mfp::note::get -item_id $new_id -array note_array}]
                aa_false "After deleting a note, retrieving it fails" [expr $get_again == 0]
            }
    }