New Demonstration Notebook v1.12

Electronic Notebook Project

Team: Al Geist, Jens Schwidder, David Jung, and Noel Nachtigal



What is an Electronic Notebook?
An electronic R&D Notebook is the electronic equivalent of a paper research notebook. Instead of recording information on paper, the sketches, text, equations, images, graphs, signatures, and other data are recorded on electronic notebook "pages", which can be read and navigated just like in a paper notebook. Instead of writing with a pen and taping in images and graphs, reading and adding to an electronic notebook is done through a computer and can involve input from keyboard, sketchpads, mouse, image files, microphone, and directly from scientific instruments. Electronic notebook software varies in how much it "looks and feels" like a paper notebook, but all the basic functions of a paper notebook are present. In addition, electronic notebooks allow easier input of scientific data and the ability for collaborators in different geographic locations to share the record of ideas, data and events of the joint experiments and research programs.

Paper notebooks are ubiquitous in the scientific community. Researchers keep personal notebooks to record their ideas, meetings, and experiments. The contents of these notebooks is usually kept private unless needed to prove when an idea was first recorded. Notebooks are kept on all major scientific instruments as well. These notebooks are shared by all the researchers that use the instrument and serve as documentation of the instrument's status and use.

The methods of entry in paper notebooks are common across all the different scientific disciplines. They include:

It is important to understand these methods while designing an electronic notebook to make sure the electronic version is as easy to use as the paper notebook. If the electronic notebook is too complex or too different from a paper notebook, then it is unlikely to catch on with the scientific community.

The electronic notebook is being recognized as an important tool that needs to be developed in order for scientists and engineers to do remote experimentation and collaboration. When a scientist can login remotely and control an instrument there needs to be the equivalent of the "paper notebook sitting beside the instrument" into which the scientist can record his/her use of the instrument. The notebook can also be used to check the previous and future usage schedule for the instrument.

An electronic notebook is a medium in which researchers can remotely record aspects of experiments that are conducted at a research facility. But use of electronic notebooks goes beyond just documenting use of remote instruments. They can be used as private notebooks that document personal information and ideas, or a single "project" notebook shared by a group of collaborators as a means to share scientific ideas among themselves.

Advantages of an electronic notebook over a paper notebook include:

The next three sections describe the most important features that an electronic notebook should have: ease-of-use, security, and portability. The last section describes ORNL's efforts and research to create an electronic notebook including a working notebook prototype.

Ease of Use

The first key to ease of use and wide acceptance of the electronic notebook is to not deviate too far from the functionality of the basic paper notebook. The initial goal would be to reproduce standard practice in an electronic format. Standard practice for a notebook varies from a single researcher jotting down personal thoughts to a group of scientists recording instrument settings and results. It does not make sense to burden the basic notebook design with features that may never be used and just serve to complicate the user interface.

The second key to ease of use is create an interface that provides intuitive entry of information. A researcher should be able to figure out how to use the electronic notebook without having to read a user manual. The electronic notebook should provide quick and simple methods for inputing text and sketches just like paper notebooks. It should also provide common methods of computer input such as cut/paste, drag/drop, and insert file.

The information that needs to be input into a notebook varies from project to project and researcher to researcher. The desire to have any type of information easy to enter requires that the input interface be expandable. In DOE2000 we proposed to use object oriented technology to provide this expandability. We called these NObs (notebook objects). Objects are entered into the notebook. There could be text objects, sketch objects, figure objects, file objects, video objects, etc... The notebook user or third parties could extend the type of objects supported in order to customize a particular notebook. Our present research involves using XML to define our multi-object entries.

Notebook Security

Access to a particular electronic notebook, its contents, and authentication of entries in a notebook are fundamental security issues that any notebook must address. Every notebook has a list of authorized users. Thus privacy is handled at the notebook access level. If it is a list of one, then it is a private notebook. If the list is greater than one then it is a group notebook. A group notebook may be set up for remote collaboration, or it could be used by the list of authorized users of a particular scientific instrument to document experiments. To provide non-tamperability of entries, the design of the notebook should automatically date, and digitally sign each entry, then append the entry to the last page of the notebook. A notebook page can be notarized at the press of a button. The notarization should be done electronically in a way that the the notary never sees the information on the page (which otherwise would be a potential espionage hole). Regualarly, a physical (paper) copy of a hash of all the notebook's notarized pages is notarized and kept by Records Management to provide a second level of proof that entries have not been tampered with. A single sheet of paper could contain the hashs of all the active electronic notebooks across an organization. All submissions across the Internet are encrypted with a session key that is created during the authentication phase of notebook access.

Witnessing is often mentioned as a more imporant feature than notarization. This is where an expert in the field familiar with the author reads the notebook pages and signs and dates them as being true and authentic. The notebook design needs to provide a way of adding such annotations to a notebook page.

Interoperability and Expandability

Scientists will be accessing and updating electronic notebooks from many different types of computers from laptops to supercomputers. The electronic notebook software should be designed to operate across PC, Mac, and Unix platforms. It is also desirable from a software engineering standpoint to create a single software package that is portable to a wide range of computer platforms rather than create several packages each targeted to individual platforms.

Because the needs of different projects and scientists vary, the notebook design should provide general interfaces for expansion of the functionality of the notebook both by third party add-ons, and user customization. The electronic notebook should have the equivalent of "plug-ins".

The aim is to create an expandable, interoperable electronic notebook that scientists and engineers will find more useful than their present paper notebooks.

ORNL's Notebook Design and Research

The following architectural design is based on the idea of developing a basic, practical electronic notebook that is easy to use, secure, and expandable. Cross platform execution is achieved by using Web browsers for accessing the notebooks. The architecture leverages the quickly evolving Web technologies developed for secure electronic commerce. It also leverages ORNL's experience with Java to develop interactive notebook entry interfaces.

In this architecture, the entire electronic notebook is Web based and can be accessed by any browser including PDAs. Access security is provided by password protection provided by today's Web servers.

Individual pages or entries of the notebook are never hidden from a researcher with valid access to the notebook. Data hiding could be dangerous and certainly does not foster a feeling of collaboration. Private information should be kept in a researcher's personal notebook, again mirroring the use of paper notebooks. It is a simple proceedure to transfer information from one electronic notebook to another if desired.

Design of our notebook software permits personal notebooks to run stand-alone on laptop computers by installing a small httpd daemon, or on desktops connected to a network. Laptops can be connected to the network and pages or pieces of pages can be cut and pasted into other elelctronic notebooks. Over a hundred such personal notebooks are in use by researchers across the country utilizing our prototype software.

Being Web based, any number of authenticated collaborators can read or add to the notebook at the same time.

Each entry made to the notebook becomes a separate HTML page that is automatically appended to the end of the notebook. As such, the entries can contain any item that a Web page can contain. In the simplest case, this is text and graphics, but the entry is not limited in any way and could contain a full complement of interactive multimedia objects, as allowed by the HTML standard. There is also a provision in the design to be able to append additional annotation information to an existing page.

Reading and writing to a notebook requires no special features of the browser. Notebook data is automatically handled by Perl5 scripts running at the site where the particular notebook data is stored. The scripts are designed to hide the location of the actual data to deter unauthorized access. Interactive input into the notebook is accomplished through CGI scripts. By using web browsers as the user interface, the ORNL notebook accessable from any hardware that runs a web browser. The software itself runs on a web server installed anywhere on the Internet, or even on a stand-alone laptop.

The header of each page of the notebook displays the notebook title specified by the user during installation, the page number and the total number of pages in the notebook. In the figure below of page 2 the title is "V1.12 Test notebook". Below the header are two rows of function buttons described in more detail below. Under the function buttons is a pull down list showing the modification history of this page. This pull down shows the date and time the page was added and each time it was edited and by who.

The present version of our notebook provides the following functions divided into two classes: those that modify the notebook and those that just view pages in the notebook.

Modifying Functions

Add - brings up an input form. entry of name and title fields is required. User can cut and paste html source or text into the text input field. By default notebook assumes plain text was typed into the input field and it preserves its formating. Once the input is complete, select the appropriate MIME type at the bottom of the form and press submit. This entry will be added to the end of the notebook.
Instead of using the text input field, the user may upload a file into the notebook, for example a gif or jpg image. After submission, the notebook will return the URL of this image so it can be linked into or displayed in a notebook page.
Our research has experimented into incorporating a Java applet sketchpad and pen input. The sketchpad permits inserting images that can then be sketched over, highlighted, or annotated. We have gotten numerous reports that our original java sketchpad no longer works correctly with the latest IE and Netscape browsers. We have started development of a new java sketchpad and have suggested that users use their favorite graphic tool such as powerpoint along with the notebook's image upload feature to get sketches into the notebook.

Edit - brings up the same input form as Add with the text input field filled with the contents of the presently viewed page. The user can edit the contents (for example to correct spelling or formatting) or add additional information. Select MimeType "html" and press Submit to put this edited page back in the notebook. We plan to add a revision log to enties to track who and when entries were altered.
[Notarized entries can not be edited or deleted]

Delete - deletes the presently viewed page from the notebook. It is equivalent to tearing out a page of a paper notebook. The original page is not actually deleted and is still preserved inside the notebook, but would require a system administrator to manually restore it.

Annotate - allows the user to add entries to the bottom of the currently viewed page, for example, if a later page contains data that supersedes or invalidates the present page. Annotate brings up an input form similar to edit except it does not allow the original contents to be changed. Annotation is allowed on pages with Notarized entries. (The annotations themselves can be notarized)

Notarize - electronically notarizes the viewed entry. A trusted time and notary server is sent a hash of the entry. The server time-stamps and signs the hash and securely returns it to the notebook, which stores this notarization with the entry. Weekly a physical paper print out of the hash value of each notebook is sent to Records Management to be notarized and stored.
[presently under development]

Navigation Functions

First - show first page of notebook.
Previous - show page previous to present page.
Next - show next page.
Last - show last page entered in the notebook. (default on startup)
Contents - show the table of contents for the notebook.
Search - brings up a form that accepts a search string.
Returns a table-of-contents-like page listing the notebook pages containing matches.

The notebooks can be setup to allow some groups to only read a notebook while other groups could have full read and write access to it.

In our design, the notebook's table of contents is automatically updated to reflect the new page and keywords describing its content. Secure Web server technology will be used to provide the secure transmission of notebook information over the internet. Again the key to our design is to leverage existing technology as much as possible and not to reinvent the wheel.

It has come to our attention that the Fnord web site mentioned in the enote documentation is no longer active. We have archived a copy of the Fnord web server software for those who wish to try it. Here are some configuration instructions for Fnord. Many other free web servers are available on the Web, we have tested the enote software with several that can be found on tucows software repository site.

If you try out the demo and would like to get a copy of this software to set up an electronic notebook for your own research project, then send me

New features in Version 1.12 (Released January 2003 - last updated Oct 2004)

New features in Version 1.11 (Released September 2001)

Features in version 1.10 (Released February 1998)

Presently, there are over 1000 groups around the country using our electronic notebook software for their projects as well as giving us feedback on desired functionality. Examples include the set of notebooks used by the national Scalable Systems Software Center, the Advanced Photon Source at Argonne, Diesel Combustion Collaboratory, the Radiation Safety Information Computational Center RSICC set of notebooks. and the set of notebooks used by the DOE Genomes to Life group (password protected), who are developing advanced supercomputing techniques to characterize all the molecular machines inside a single celled organism.

The notebook software, which is written in Perl5, works on Win XP, 2000, Me, NT, 98, Mac, and Unix systems. It requires only that a web server and Perl5 be installed on the machine where the notebook resides.

A demonstration of the notebook interface is available online to try out and see if it meets your needs. If you try out the demo and would like to get a free copy of this software to set up an electronic notebook for your own research project, then send me email:

Our work was sponsored by the DOE2000 Electronic Notebook Project. Beyond enote1.12, we have recently been working on a version of enote that is written completely in Java rather than Perl. Called "jenote" this version is presently in beta test by scientists at the Spallation Neutron Source facility.

Last Modified February 24, 2010 by Geist