Running example

This tutorial uses a running example of a personal photograph library, starting with a simple catalogue of images and some associated metadata, and then adding additional structures to cross reference other entities involved in the picture-taking process. For this example, we use a prefix pcat for URIs (actually ["Compact URIs"](http://www.w3.org/TR/curie/), or CURIEs) defined specifically for the photograph catalogue. This prefix will eventually be associated with a base URI to allow identification on the World Wide Web. (@@TODO: full linked data support.)

The completed example can be seen at the Annalist demo site, as collection [Annalist photo collection]. (@@TODO, once current in-development software is deployed.)

Log in to Annalist

Browse to the Annalist site login page; e.g. http://demo.annalist.net/annalist/site/, then http://demo.annalist.net/annalist/login/.

The first time of logging in to an Annalist site from some computer will involve some or all of the following.

  • select a User ID

  • enter User ID into login form

  • select "Google" for the Login Service

  • click "Login"

At this point the browser is redirected to Google’s own login page to enter account credentials, and maybe also a 2-factor authentication code. Once the authentication credentials have been entered and verified, Google may then ask for your permission to disclose information to the Annalist application. The information to be disclosed is described; Annalist uses only the email address, which is used in conjucntion with the User ID entered to select a set of user permissions (see "User permissions" section).

@@When alternative ID provders are supported, note this

If the User ID has already been used by a different user, a message is displayed (@@ message text)

After the first time, use the same details and click "Login". Typically, if this is a personal machine you have used before, the login will complete without any further interaction.

Annalist login form
Figure 1. Annalist login form

The login sequence associates an authenticated email address with the User ID given, which is used to determine appropriate access permissions (see "User permissions" below).

Annalist logged in
Figure 2. Annalist logged in

Once the login is complete, click on "Home" in the menu bar to return to the main Annalist site

Annalist home page
Figure 3. Annalist home page

Using public computers

If you login from a public computer, remember to log out from your identity provider when you have finished using Annalist and before leaving the computer.

@@details for logout from Google.

Local authentication

@@Update this when "Local" is implemented as a login service

Annalist has been designed to use third party authentication. This avoids having to deal with the tricky technical and operational difficulties of managing password files; leaked password files are a very common form of security failure, so by leaving this to the ID provider professionals, user security should be improved. This also allows users to log in using an existing account rather than having to allocate and remember a new password for Annalist.

Sometimes (for example when Annalist is required to be usable when not connected to the Internet), it may be preferred to use locally managed passwords. Annalist can use the local password management and login features of the Django web application platform on which it is built.

When installing Annalist, an administration account may be created using the annalist-manager tool. When logged in to Annalist using this account, the Admin link in the footer of most Annalist pages will allow new user accounts to be created via the Django admin interface. More documentation about using this admin interface is in the The Django Admin Site, which is Chapter 6 of The Django Book.

To log in using a Dango account clock on the "Local user credentials" login link, and enter a Django username and password into the new page that is displayed. Once logged in, return to the Annalist site "Home" page

Annalist local login form
Figure 4. Annalist local login form

User permissions

Annalist keeps a set of user permissions associated with every combination of User ID and authenticated email address.

User permissions may be defined site-wide (i.e. they can apply for all collections across a site), or they may be defined separately for each collection. Thus, different permissions may be granted to different users in different collections. Permission to create new collections must be site-wide.

Within a collection, a user with ADMIN permissions on that collection can view the user permissions for the collection in the List view List users. To view the list of user permissions for a collection, browse to that collection which will display some list of records. In the List view dropdown, select option List users, and click View see see user permissions defined for the collection only, or View all to also list site-wide user permissions.

Annalist collection user permissions
Figure 5. List user permissions for an Annalist collection

Special Annalist user identifiers

There are two "special" users that are used to select access permissions applied to users for whom no more specific permissions are available:

annal:User/_unknown_user_perms defines permissions that are applied for any user who is not logged in to Annalist. Typically, this might be used to grant public read-only access to a collection.

annal:User/_default_user_perms defines permissions that are applied for any user who is logged in to Annalist, but for whom no more specifiuc permissions are granted. Permissions thus granted are effectively available to anyone who comes to the web site, but any actions they perform are potentially auditable, being associated with an authenticated (by Google) email address.

Adding user access permissions to a collection

New permissions may be added from the "List user permissions" page (see above) by clicking New:

Annalist collection new user permissions
Figure 6. Creating new user permissions for an Annalist collection

The User Id field corresponds to the User Id entered in the login page.

The URI field (usually a mailto: URI) corresponds to the authenticated email address provided by the identity service (e.g. by Google). In principle, other forms of URI might be authenticated by an ID service, but these are not currently part of the OpenId Connect protocol used.

The Permissions field is a list of tokens (names) corresponding to permissions granted to this user. For the permissions to apply, both the user ID used to log in and the authenticated email address must match the User Id and URI fields.

Adding site-wide user access permissions

Site-wide permissions are initially defined using the annalist-manager command line administration tool. See Installing and setting up Annalist for more information about annalist-manager, or run the command annalist-manager help.

Once basic administrative access has been established, additional site-wide permissions can be created by creating or editing user permissions in the _annalist_site collection.

@@Update with more detailed instructions.

@@analist-manager currently has a bare miniumum of capabilities to create users, mainly intended to bootstrap a system with admin users.

Annalist permission tokens

The standard Annalist permission tokens include:

ADMIN - required to create or view user permissions in a collection. The creator of a collection is automatically granted ADMIN permissions over that collection, so they can assign permissions in that collection for other users. This permission at site level also allows creation and deletion of collections.

CONFIG - required to change the structure of a collection: to create and/or modify record types, views, lists, etc.

CREATE - required to create new data in a collection.

UPDATE - required to edit data records in a collection.

VIEW - required to view or read data records in a collection.

DELETE - required to remove data records in a collection.

CREATE_COLLECTION - this permission, or ADMIN, is required at site level to create a new collection.

DELETE_COLLECTION - this permission, or ADMIN, is required at site level to remove an existing collection.

(Future developments may allow for the introduction of additional tokens on a per-collection basis, but for now these are all of the available permissions.)

Browsing an existing Annalist collection

@@TODO

@@Top bar (Home, collection, type)

@@Bottom bar (About, Contact, Sitemap, Admin)

@@Other common controls: Choose view; List view / view all; Customize; Set default

Creating a new collection

Creating a new collection requires site-level permission CREATE_COLLECTION or ADMIN.

To create a new collection, go to the Annalist site home page and enter a collection id and description for the new collection, and click the New button. The collection id must consist of letters, digits and underscore ('_') characters only, and be no longer than 32 characters.

Creating a new Annalist collection
Figure 7. Creating new Annalist collection

In the updated list of collections, click on the link corresponding to the new collection to view its initial content:

Creating a new Annalist collection
Figure 8. View new Annalist collection

Change collection metadata

Modifying collection metadata requires CONFIG permissions.

A collection name, label, description and other metadata may be edited by viewing the site home page, selecting the checkbox by the collection to be edited, then clicking on the "Edit selected" button:

Edit Annalist collection metadata
Figure 9. Edit Annalist collection metadata
Annalist collection metadata edit view
Figure 10. Annalist collection metadata edit view

@@TODO: update home page screenshot from demo site when deployed

By default, a collection inherits site-wide type, view and field definitions that are defined by the Annalisty software. It may also inherit definitions from another existing collection on the same site by selecting that collection in the Parent field. This feature is intended to allow a collection to be based on an existing set of definitions rather than starting from scratch.

Create initial record type and views

It is possible to start creating new data records straight away in a newly created collection, using the in-built default type and views. But for practical use, it is probably better to start by creating a new record type and corresponding views to match the initial data to be collected.

Creating record types and views for a collection requires permission CONFIG in that collection. The user who creates a collection is automatically granted full permissions for that collection. They may, in turn, create permissions for other users.

Switch to a listing of record types, by selecting List types from the List view selector, and then click on the View button:

List record types in Annalist collection
Figure 11. List record types in Annalist collection

Click the New button to create a new record type, and enter some details about this record type (e.g. as shown below). To define an initial tailorable view and list descriptions for the new record type, click on the Define view+list button. The Default view and Default list fields are updated accordingly:

Creating new record type in Annalist collection
Figure 12. Creating new record type, view and list in Annalist collection
Note
@@FIXME

There is a possible bug in the software that needs investigating: if an error message is displayed saying the new type already exists, click Cancel then select the new type in the "List types" display and click Edit. Then click the Define view+list button.

In this case, the automatically generated label, "List of Photograph", doesn’t really read smoothly. Details of the generated list and/or view descriptions can be edited by clicking on the nearby button-edit-entity button, and making changes as desired. Click Save when done to return to the type description page.

Edit definition for list of photographs
Figure 13. Edit definition for list of photographs

Click Save again to save any final changes to the type description and return to the "Type list" display.

Create some initial data records

Switch to a listing of "Photograph" records by selecting Photographs from the List view selector, and then click on the View button; the initial list should be empty:

Initial
Figure 14. Initial, empty, list of Photographs

Click on the New button to bring uo an initial forkm for enterimng details about a photograph. New record views contain a number of common fields: Id, Type, Label and Description. Only the Id field and Label fields are mandatory; the others are defined because they are commonly useful. The Id field is used internally to identify the record, and is used in forming a URL that can be used to access the data. The Label field provides a summary description of the record used in lists and drop-down selectors.

Enter some descriptive data into the fields and click Save.

Creating new Photograph record in Annalist collection
Figure 15. Creating new Photograph record

This process can be repeated for additional photographs.

List of Photograph records in Annalist collection
Figure 16. List of Photograph records

Add simple fields to a data record

The default view fields presented when creating new records suggest a minimum amount of information to appear in a record. For many practical purposes, additional fields will be required. In the case of a photograph, one might wish to have separate fields to record when and where it was taken. New fields can be added to a view at any time by editing the view description.

There are several ways to edit a view description:

  • Select List views in the List view dropdown, click View, then select the view to edit from the displayed list and click Edit

  • Click Customize on any list display, select the view to edit from from the column headed Record views, and click on the Edit button below.

  • Display an instance of the view to be edited (e.g. a view of one of the Photographs in the list of photographs) by clicking on an Id link in the list, then click on the View description button and finally click on the Edit button of the pages displayed.

The following screenshots illustrate the last of these approaches.

View a Photograph record in Annalist
View description of view of photograph in Annalist
Edit description of view of photograph in Annalist

To add a "Date" field: click Add field. A new row is added to the list of fields, with unspecified values for Field id, Property and Position/size. Clicking on the Field id value lists a few options, but none of these suggests a "Date" value. A new type of view field is required here, so click on the button-new-entity button:

Adding a new field to the Photograph view

This opens a new form to define details of a new field type. Fill in information for Id, Field value type, Label, Help, Property, Field render type, Value mode and Placeholder as shown. In ths case, the other fields should be left unspecified. (The meanings of all these fields are described in View fields in Annalist.)

Defining a new view field type for date
Figure 17. Defining a new view field type for date

When done, click Save, which returns to the previous view editing form. Now, clicking on the Field id value includes an option for the new field type just described. Choose this. The Property and Posityion/size columns can be left blank to use values from the field definiotion, or overriding values for the current view can be specified here. Select an appropriate value (0/6) for Position/size.

Repeat the above process, starting with Add field, to define a new field for the location that a photograph was taken:

Defining a new view field type for place
Figure 18. Defining a new view field type for place

Returning to the view edit form, the new location field can be selected and its position/size specified. Next, select the two new fields and click the Move ⬆ button so that they appear immediately after the label field.

New fields added to Photograph view
Figure 19. New fields added to Photograph view

Click Save, then Close to return to the view of a Photograph, which should now look like this:

Updated view of a Photograph record in Annalist
Figure 20. Updated view of a Photograph record in Annalist

If the view is now edited, values for the date and location fields can be entered:

Adding new field values to a Photograph record
Figure 21. Adding new field values to a Photograph record

These new fields are clearly intended to hold specific types of value (date, location) and the examples suggest particular formats be used for them. But as far as Annalist is concerned, these are just simple text fields, and no attempt is made to check the format of any values entered. This is consistent with the Annalist pholosophy of making it easy to capture whatever data may be available with a minimum of hindrance to the user. The intent is that issues of consistency and data quality may be checked separately according to whatever criteria are deemed appropriate to the task at hand.

Upload images to a collection

A glaring omission from the record of a photograph created thus far is the photograph itself. Annalist supports a notion of "attachments", which are arbitrary files that are stored with an Annalist data record, and are made visible through appropriately defined fields, and which may also be accessed directly by Annalist-mediated URLs. Field definition options allow attachments to be uploaded via the browser from the user’s local file system, or imported from a web site. This example uses file uploads.

The steps for adding an image attachment to a record are:

  1. Define a new field type for the upload imaged file

  2. Add the new field to the Photograph view description

  3. Edit Photograph records and upload images

There are several ways to accomplish these steps (see previous section). The following example goes via the "Customize" page, starting from the "List of photographs" page:

List of Photograph records in Annalist collection
Figure 22. List of Photograph records

Click on the Customize button:

Annalist collection Customize page
Figure 23. Customize collection page

Create a new field definition for images

Select "View of Photograph" in the "Record views" column, and click the Edit button. On the resulting view description page, click Add field, and then on the + button beside the newly added field:

Add new field to photograph record
Figure 24. Add new field

Fill in details for the new field as shown:

New image field details
Figure 25. New image field details

The key fields to note here are:

  • Render field type: the value Ref image file indicates the field value is a reference to an image file.

  • Value mode: the value File upload indicates the referenced image will be an uploaded file.

With the field details entered, click 'Save' to return to the view editing form.

Add new image field to view description

Select Field Id Image for the newly added field, select a value for size/position, click Save to return to the Customize page, and Close to return to the list of photographs:

Photograph record with
Figure 26. New "Image" field added

Upload image to Photograph record

To upload an image, click on the link for a photograph, then click Edit; the photograph editing form, is displayed, now with an additional Image field with a Browse button:

Edit Photograph record with Image field
Figure 27. Edit photograph record with Image field

Click Browse, select an image file to be uploaded, then Open (or equivalent for the browser being used). On the photo editing page, click Save. The Photograph record is now displayed with the uploaded image:

View photograph record with uploaded Image
Figure 28. View photograph record with uploaded Image

Clicking on the displayed image here will show the image alone in a new tab (or possibly a new window, depending on the browser used), occupying the full browser window.

Add repeating fields to a data record

Sometimes, it is desirable to have a field or group of fields in a record that can be repeated an arbitrary number of times. We have already seen this when editing a view description which may contain an arebitrary number of fields. Annalist implements such repeated fields as a special type of field that itself contains references to other fields via a "Field group" description.

Thus, to create a repeating field or group of fields within a view, the following must be defined:

  1. One or more ordinary individual fields that are to be repeated.

  2. A field group that references the field(s) that are to be repeated.

  3. A repeating-value field that references the field group.

Annalist provides a short-cut for creating these various descriptions in the form of a "task button" that appears on the field description editing form.

For our example, we create a field that allows multiple keywords to be associated with a Photograph, starting with a view of a photograph record. Click on View description, then on the next page displayed click Edit, Add field, and then on the button-new-entity button by the newly added field.

Now fill in details for a single keyword field, as shown. When the details have been entered, click on Define repeat field.

New keyword field details
Figure 29. New keyword field details

Now click on Save, and select Repeat field 'Keyword' for the new field id.

Photograph view with
Figure 30. New "Repeat field 'Keyword'" field added

The repeat field structure just created is perfectly functional, but the automatically generated label field could be improved. Click on the button-edit-entity button by the "Repeat field 'Keyword'" id. Change the Label and Placeholder fields as shown, then click on Save:

Edit labels used with
Figure 31. Edit labels used with "Keyword_repeat" field

On reeturning the the view editing form, noteice that the selection label for the newly added field is changed to "Keywords":

New field label updated
Figure 32. New field label updated

Click Save then Close to return to the Photograph record view. To add some keywords, click Edit, then Add keyword, and fill in key word or phrase text. repeat for as many keywords as desired:

Edit photograph record keyword fields
Figure 33. Edit photograph record keyword fields

Click on Save to view the resulting record:

View photograph record keyword fields
Figure 34. View photograph record keyword fields

Create an additional data type (with view+list)

The examples so far have been based on a single "Photograph" record type. Many interesting data collections consist of multiple cross-referenced record types. For example, we can create "Location notes" records to hold information about where photographs were taken.

Start by creating a new Type, with corresponding View and List definitions; e.g.

  1. Click on Photo_collection in the top menu bar

  2. Select List types from the List view drop-down, then click View

  3. Click on New to create a new Type

  4. Enter details as shown

  5. Click Save, then select the newly created type and click Edit. (@@This step is a workaround for a bug in the Define view+_list handler, and should not be needed.)

  6. Click on Define view+list

  7. Click on Save

Create new type Location_notes
Figure 35. Create new type Location_notes

Next, edit the Location notes view to include a map reference field:

  1. On the List types page, click on the link Location notes.

  2. Click on the link View of Location notes

  3. Click Edit

  4. Click Add field

  5. Click the + button by the newly added field

  6. Fill in details of the new field as shown.

  7. Click Save

  8. Select Map reference for the field id of the newly added field, and a value for Position/size.

  9. Click Save, then Close to return to thelist of views.

Now we can create an instance of the new type:

  1. Click on Photo_collection in the top menu bar

  2. Select List of Location notes from the List view drop-down, then click View

  3. Click on New to create a new Location notes record

Create new map reference field
Figure 36. Create new map reference field

A similar process is repeated, but this time also using the Define repeat field button, to create a repeating field of links to further information about the location:

Create new web link field
Figure 37. Create new web link field
Further information field
Figure 38. Further information field

Now the new fields can be configured in the Location_notes view:

Location notes view with map reference and further information fields
Figure 39. Location notes view with map reference and further information fields

A Location notes record can now be created with data for the additional fields:

Create a location notes record
Figure 40. Create a location notes record

The preceding section created a new record type for location notes. Here, we edit the Photograph view to allow each photograph to be linked to notes about the location where it was taken. This will be achieved by changing the type of Location_taken field from a simple text field to a reference to a Location_notes field.

To do this, open up the form for editing the view description Photograph (using any of the navigation paths described previously). Then click on the writing hand button beside thefield id "Location taken":

Edit field
Figure 41. Edit field "Location taken" in Photograph view

Edit the "Location taken" field as shown beloiw, noting particularly values entered in these fields:

  • Field value type - same as the URI given in the Location_notes type record (currently not used other than for documentary purposes)

  • Field render type - Optional entity ref presents a dropdown of entities to which the field may link

  • Value mode - Entity reference indicates this field is a reference to some other Annalist entity in the current collection.

  • Refer to type - Location notes indicates the type of entity to which this field may link.

Edit field definition for Location taken
Figure 42. Edit field definition for Location taken

Save the updated field and view descriptions, and redisplay one of the Photograph records: note that the Location taken value is still displayed, but is rendered in a style used for non-existent entity references. Click on Edit to edit the record data, and select the vcalue Sileby Mill from the dropdown (corresponding to the previously created Location notes record).

Select location notes record from dropdown
Figure 43. Select location notes record from dropdown

Click Save to return to the Phbotograph view, and note that the Location taken field now displays as a link to the selected Location notes record

Display fields from a linked record

Rather than just a link to a related record, it is sometimes useful to display one or more values directly from such a record. For the present example, the photograph "location taken" field is modified to display a link, description and map reference for the location in the photograph view.

This kind of display is created using a Fields of referenced entity field type. This is a special kind of view field that behaves very differently in edit and view modes: in edit mode, it works just like an Optional entity ref field, allowing the user to select an instance of some designated type. But in view mode, it displays one or more fields from the referenced entity.

First, navigate to a field description in the Location_notes view; e.g. from any photograph view, click View description, Location taken, Location notes, View of location notes Map reference and Edit.

View of photograph View of photograph view description View of location taken field description View location notes type description View location notes view description View map reference field description

Edit map reference field description form
Figure 44. Navigate to edit form of "map reference" field

An alternative to this slightly long winded navigation path would be to go to the collection front page (click Photo_collection in the menu bar), select List fields from the List view selector, cick View, click on Location_mapref. Either way, the resulting page should look something like the last screenshot above.

In the form displayed, click Define field reference: this will create a new field and field group Location_mapref_ref referencing this field, and display an editing form for the new field definition. Before making any changes to the field itself, click on the button-edit-entity button by the Field group field:

Edit location notes reference field
Figure 45. Edit location notes reference field

This will display a new form for editing details of the new field group. In this form:

  1. Add additional fields Id and Label to be displayed, using the Add field button and the new field id selectors dislayed.

  2. Move the two new fields in front of the map reference field by selecting their checkboxes and clicking Move ⬆ button.

  3. Make changes as desired to the field label and description (but leave other fielkds as they are).

  4. Click Save to save the changes and return to the new field definition that references this group.

Edit new field group
Figure 46. Edit new field group

Back in the field editing form, change the Id, label and description and property URI of the new field to better reflect its actual usage. Then click Save.

Edit new location notes reference field
Figure 47. Edit new location notes reference field

At this point, navigate to the view description for Photograph (e.g. by displaying a Photograph record and clicking on View description, or selecting and displaying List views from the collection home page and then clicking on Photograph). Then click on the Edit button to display the view editing form. On this form, clicik Add field, select the newly created Location notes field type, and select 0/12 for the position/size value:

Add location notes field to photograph view
Figure 48. Add location notes field to photograph view

Click on Save.

Now navigate to a view of a photograph record, click Edit and inthe displayed form select a value for the new field Location notes from the dropdown displayed.

Select value for Location notes field
Figure 49. Select value for Location notes field

Click Save to view the photoigraph records, and observe that the Location notes field now displays three values from the selected record:

View photograph record with location notes
Figure 50. View photograph record with location notes