What is Web 2.0 ?

Web 2.0 is a phrase coined by O'Reilly Media in 2003 which refers to a perceived second generation of web applications such as social-networking sites, wikis which facilitate collaboration and sharing between users. The article published by Tim O'Reilly explains that the concept of Web 2.0 does not have a hard boundary but can be visualized as a set of principles and practices.

Features common to most web 2.0 applications

Foloksonomy: Tagging

A folksonomy is a user-generated taxonomy used to categorize and retrieve content such as Web pages, photographs and Web links, using open-ended labels called tags. Digital content such as photographs cannot be retrieved by searching for a phrase unless the photograph itself is tagged with phrases.

Collaboration or Participation

Sharing

Social networking sites allow members share content like blogs, photos, or event interests with their friends to keep participation levels high.

Commenting and Voting

Users of Web 2.0 web applications add value by commenting (Amazon Reviews) and voting on commodities displayed by the web application.

RSS

Web sites can publish event summaries of interest as feeds in popular formats RSS & Atom. These feeds can be subscribed by users using their browsers or third party aggregators like Google Reader. Instead of signing up for a 'newsletter' users can choose to track information by subscribing to published feeds.

Rich User Experience

Rich User Experience which employs AJAX techniques is hallmark feature of any Web 2.0 application. Dekoh platform provides widgets which can be re-used in applications.
Dekoh Widgets

• Refer to web widgets
• Refer to DWR

Refer to Prototype

Overview of DataModel

The common Web 2.0 features are abstracted in Dekoh platform API. This includes data model, UI widgets and other utilities. The Data model is backed by a Database. The mapping from Java objects to relational database is achieved by employing a Java Persistence API implementation (Toplink-Essentials which is distributed by open source project Glassfish).

DigitalObject

DigitalObject (also called Dio) provides the frequently used Web 2.0 features like tagging, commenting, rating. Dekoh application developers should identify object abstractions in their application which need all these features. Such identified objects can extend DigitalObject class to inherit the Web 2.0 features.

Following pseudo code outlines some of the fields in DigitalObject.

Class DigitalObject 
{ 
int id; 
String name; 
String description; 
Rating rating; 
Set<DIOProperty> dioProperties; 
Set<Tag> tags; 
}

Application developer can choose to extend DigitalObject and further define properties and their data types. Alternatively developer can also instantiate DigitalObject and customize it by adding properties (name-value pairs) of String type.

Following code snippets contrast these two approaches for a potential implementation of Book and Photo as a DigitalObjects. Consider the following code snippet containing definition of Book for a application that lets users create book catalogs and share their favorite books. The approach uses the Properties on DigitalObject to persist information about a Book's Author, Title and ISBN etc.

public class Book
{
    private static final String AUTHOR_PROPERTY_NAME= "books.authorName";
    private static final String TITLE_PROPERTY_NAME= "books.title";
    private static final String ISBN_PROPERTY_NAME= "books.isbn";

    private final DigitalObject delegate;

    public Book(String author, String tile, String isbn, EntityManager
em)
    {
        this.delegate = new DigitalObject("books");
        this.delegate.setOrAddDioProperty(AUTHOR_PROPERTY_NAME, author)
                .setOrAddDioProperty(TITLE_PROPERTY_NAME, tile)
                .setOrAddDioProperty(ISBN_PROPERTY_NAME, isbn);
        em.persist(this.delegate);
    }


    public String getAuthor()
    {
        return this.delegate.getDioProperty(AUTHOR_PROPERTY_NAME);
    }

    public String getTile()
    {
        return this.delegate.getDioProperty(TITLE_PROPERTY_NAME);
    }

          public String getIsbn()
    {
        return this.delegate.getDioProperty(ISBN_PROPERTY_NAME);
    }
}

Notice that none of these fields need to be mapped to a Database Table. DigitalObjects internally saves the all the DioProperties set on any instance to a Table.

However this simplicity comes at the cost of executing a query to find values for properties author, title, isbn etc. from the DigitalObject.

Contrast the above approach with the code below. The snippet employs for defining Photo for an application that manages the photos taken by user and share the photos.

@Entity @SecondaryTable(name = "PHOTO_TABLE") Class Photo extends DigitalObject {
  URI fileLocation;
  public Photo(URI fileLocation) {
    this.fileLocation = fileLocation;
  }
  @Basic
  public URI getFileLocation() {
    return fileLocation;
  }
}//end of Photo class
//Creation of Photo elsewhere 
URI photoUri = new java.net.URI("C:\\Documents And Settings\\me\\My Documents\\My Pictures\\DSC0074.jpg"); 
Photo aPhoto = new Photo(photoUri); 
entityManager.persist(aPhoto);

In this approach Photo entity extends DigitalObject to define a attribute called fileLocation which is of java.net.Uri type. This new entity needs to be mapped to a database Table using Java Persistence API annotation. Any fields that need to be persisted should be mapped to this columns of this Table using JPA annotation Basic or Column.

DioCollection

A DioCollection is a persistent collection (a set, duplications are not allowed) of Digital Objects. The DioCollection itself extends DigitalObject, and hence inherits properties of a DigitalObject. By virtue of thie extension any DioCollection instance can be tagged, and rated.

Tag

Tag is a word or a phrase that describes an item. Tags are typically chosen by the owner of the content. Once content is tagged, owner can retrieve the relavant content by searching by the tag. Using Dekoh platform API each application can compose its own vocabulary of Tags. DigitalObject instances can be tagged. For a given word or phrase, there must only be one Tag entity in the Database. So application developers should use factory method getOrCreateTag() instead of using the constructor.

Tag tag = Tag.getOrCreate(entityManager, 'vacation',applicationId); 
//Prefered way to create and persist a tag.

Since Tag is a Java String, it can have spaces or commas and other characters that are usually treated as separators and not actual legal characters in a Tag. Dekoh Applications should define which character they will use for separator, and perform validations on user input.

For each word which is persisted as Tag entity, all the DigitalObjects that have been tagged with that word can be retrieved. In other words Tag and DigitalObject share a many-to-many bi-directional relationship.

Comment

Abstracts a node in threaded comments. Each CommentNode has a parent and list of Children. First comment in a Thread will not have a parent.

public class Comment {
    private int commentId;
    private Buddy commentor; 
    private String comment;
    private Timestamp postedAt;
    private List<Comment> replyComments; //Can be null if there no replies.
    private Comment parentComment; //Can be null if this is the first comment
    private DigitalObject commentedOn;
}

LocalUser

LocalUser is a Dekoh user who logs into the Dekoh Desktop.

Directory and FileLocation

For DigitalObjects that represent a file on a computer like a JPEG Photo, or an mp3 music file, Directory and FileLocation classes can be helpful.

Persistence API

Dekoh platform's data model uses Java Persistence API to save the DigitalObjects to a Database. Every Dekoh installation includes Apache Derby database, and Java Persistence implementation called Toplink Essentails . Dekoh applications can use Java Persistence API in "standalone/Java SE" mode. Apart from using the JPA annotation to declared entities and the Object-Relational mapping, using JPA requires applications to define a "persistence unit".

Defining a persistence unit involves writing a deployment descriptor called persistence.xml. In this XML the entity class names are identified. Each persistence unit is given a name. Application can later use the declared persistence unit name to obtain instance of EntityManagerFactory.

<persistence xmlns="http://java.sun.com/xml/ns/persistence"
version="1.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd">
    <persistence-unit name="my-persistence-unit">
....
</persistence>
EntityManagerFactory emf = Persistence.createEntityManagerFactory('my-persistence-unit',
properties);

Persistence unit defination is also expected to include, details about how to connect to Database. Dekoh applications should create this database in the 'user-data' directory that is present in each Dekoh installation. On windows the user data directory will be at C:\Dekoh\user-data, on unix based computers this will be at ~/Dekoh/user-data. For Dekoh applications to remain portable location of this Database cannot be hardcoded in the persistence.xml.

Since Dekoh applications are web applications initializing the Persistence Unit can be done in a ServletContextListener. Following method sets up the application database in Dekoh's user data folder.

/**
     * Initializes the persistence unit of this application. <tt>persistence.xml</tt> must be present in META-INF dir in
     * the classpath. All entity classes identified in the persistence.xml should be available in the classpath.
     *
     * @param dekohComponentId as written &lt;id> field of this application's component.xml
     * @param puName           persistence unit name : value of name
attribute of &lt;persistence-unit> in this
     *                         application's persistence.xml
     * @return EntityManagerFactory from which entity Managers can be created. Store this reference in servlet context
     * @throws IOException
     */
private EntityManagerFactory initializePersistenceUnit(
String dekohComponentId, String puName) 
throws IOException
    {
        PortalApplicationContext applicationContext = PortalApplicationContext.getContext(dekohComponentId);
        File applicationDataDirectory = applicationContext.getApplicationDataDirectory();
        File databaseDirectory = new File(applicationDataDirectory,
"db");        
        String jdbcUrl = "jdbc:derby:" +
databaseDirectory.getAbsolutePath() + ";create=" + !databaseDirectory.exists();
        Properties databaseProperties = new Properties();
databaseProperties.setProperty(oracle.toplink.essentials.config.TopLinkP
roperties.JDBC_URL, jdbcUrl);
databaseProperties.setProperty(oracle.toplink.essentials.config.TopLinkP
roperties.JDBC_DRIVER, "org.apache.derby.jdbc.EmbeddedDriver");
        dekoh.portal.fwk.jpa.EntityManagerFactoryOrganizer eo = EntityManagerFactoryOrganizer.getInstance();
        javax.persistence.EntityManagerFactory emf = eo.getOrCreateEntityManagerFactory(puName, properties);
        return emf;
    }

Because of a bug in toplink-essentails, the JPA implementation that Dekoh uses more than one Dekoh application that uses JPA cannot be deployed. To workaround this bug Dekoh application should not use

 
javax.persistence.Persistence.createEntityManagerFactory(puName,properti
es)
instead use 
dekoh.portal.fwk.jpa.EntityManagerFactoryOrganizer eo = EntityManagerFactoryOrganizer.getInstance();
        javax.persistence.EntityManagerFactory emf = eo.getOrCreateEntityManagerFactory(puName, properties);

For Dekoh applications that use Dekoh platform's entity model (using DigitalObject, DioCollection) the name of the persistence unit should end with 'dio-persistence-unit'. When a persistence unit with such name is encountered Dekoh will automatically add the dekoh platform entity classnames. These entity class names need not be declared in the Dekoh application's persistence.xml.

Entity Managers and Persistence Context

A persistence context is a set of managed entity instances in which for any persistent entity identity there is a unique entity instance. Within the persistence context, the entity instances and their lifecycle are managed by the entity manager. (JPA Spec Section 5.1).

The EntityManager for a persistence context can be obtained from the EntityManagerFactory. Since When container-managed entity managers are used (in Java EE environments), the application does not interact with the entity manager factory. The entity managers are obtained directly through dependency injection. Dekoh application cannot enjoy this luxury and have to manage the lifecycle of the Entity Managers.

EntityManagerFactory once obtained is usually stored in application context, so that EntityManager can be obtained by calling

entityManagerFactory.createEntityManager();

To start a Transaction:

 EntityManager em = entityManagerFactory.createEntityManager();
 try {
   EntityTransaction tx = em.getTransaction();
   tx.begin();
   ...
   tx.commit();
 }finally(){
  em.clear();em.close();
 }

Search and Pagination

All DigitalObject that were tagged with given words can be retrieved by calling method findAllDigitalObjectsHavingTags on DioEntityManager. Java Persistence API allows one-to-many, many-to-many mappings, The collections at either end of these relationships can be marked to be fetched eagerly or lazily. When marked to fetch lazily, loading of members in relationship is postponed until the first call is made to the relationship method.

However to achieve deterministic runtime memory usage and avoid running out of heap size when relationships resulting in very large collections are loaded all at once, Pagination needs to be employed. Pagination allows to load only the specified number of objects into memory at once, thus controlling the memory usage.

Java Persistence API allows pagination to be implemented by virtue of methods setFirstResult() and setMaxResults() on javax.persistence.Query.

Dekoh platform supports retrieving a Paginated result of DigitalObject as a result of any Query. Calling paginate method on DioEntityManager. paginate() method takes the following parameters as input :

Parameters:

• getObjectsQuery: the query to be executed to get the list of entities

• countQuery: the query to get the total count, can be null if totalDios is supplied
• startIndex: index from where the results should start (starts from 0)
• pageSize: number of entities to include in a page
• mapOfParameterObject: the objects and placeholders to be set on the query
• dioClass: class of DigitalObject

Example Usage:

Page<Photo> page = paginate(getQuery, countQuery, 0, 10, null, 0,Photo.class); List<Photo> first10Photos = page.getDigitalObjects();
Page<Photo> nextPage = null
if(page.hasNext()) {
 nextPage = paginate(getQuery, page.getTotalDios(), page.nextPage(), 10, null, 0,Photo.class)  photos = page.getDigitalObjects();  
display(photos); }

Subsequent calls to paginate() method to goto next page should use the totalDios set on the Page object. Count query to figure out the total number of dios need not be executed repeatedly when scrolling to next page.

Sharing and Access Control

Dekoh applications installed on user's Desktop can be shared with contacts in the Dekoh Network. Requests to the application that is not shared to contacts will be rejected by the Dekoh Network Routing mechanism.

This level of access control to Dekoh Application is provided without need for any code or configuration from Dekoh Applications.

To start with Dekoh Applications installed on user's Dekoh installation are marked 'private' and are not shared with anyone. Using Dekoh, owner's of the content should have finer control over which content the visitors will see, and what operations they can perform on the content. Sharing a Dekoh Application is only a prerequsite so that the Dekoh network routes the request from a contact to Dekoh application.

After Dekoh network authorizes the contacts request to access the application, the request is directed to application on the owner computer. Now it is the responsibility of the Application running on owner's Dekoh desktop to perform authorization on which content the visitor has access to, and at actions are allowed to be performed on the shared content.

One of models for content sharing and Access control is like this :

• Owner collects the DigitalObjects he wants shared to a DioCollection
• Owner also chooses the Contacts he wants to share the content with.
• Owner shares the DioCollection with the chosen list of Contacts.
• Contacts whom the collection is shared to access and view it. They are allowed to post a comment, but are not allowed to modify the meta-data of the content. Contacts cannot remove, or add new tags on the DigitalObjects. They cannot remove any of the DigitalObjects.

This is the model that Dekoh platform currently supports.

Dekoh applications that intend to use this Access control model for the content they manage need to do the following.

Set up ShareFilter

ShareFilter is a HttpServletFilter which determine if the request is being made by the owner or a contact.

If the request is made by the owner, then sets up access controls such that any operations (create, read, update, delete) can be performed on the content.

If the request is known to be made by the contact then ShareFilter sets up Access Controls such that contact can view the shared content and besides posting a comment, cannot modify the database of DigitalObjects in anyway.

The contact does not have any content shared to. Thus the owner is allowed to administer the Shares and ShareFilter enforeces read-only access to the shared content to the contacts.

How is this achieved?

ShareFilter controls the lifecycle of Persistence Context (citation needed to JPA term). The transaction in this persistence context is rolled back when the request determined to be from a contact. Thus any changes to the entities in the persistence context, even inadvertently are not saved to Database. The transaction of the persistence context is committed, writing the changes to entities if the request is from the owner.

Accesspath

To allow/restrict a user to access to content, Dekoh platform need to know what is being accessed. Every DigitalObject is identified by a unique number assigned to it. Several related DigitalObjects may be grouped in a collection. Same DigitalObject may be part of sevaral collections.

Dekoh platform allows sharing any such collections to their contacts . Hence there are several contexts in which any given DigitalObject can be accessed. AccessPath unambiguously determines which DigitalObject is being accessed and in which context.

Structure of AccessPath

AccessPath is simply series of numbers representing ids of Share, Collections each separated by '-' (hyphen). The id of DigitalObject is supplied separately.

Example: AccessPath like '2-36-67' points to collection with id 67 which is part of another collection with id 36. The share being accessed is with id 2.

ShareFilter expectes to find the accesspath as value for the query string parameter 'currentpath', and the DigitalObject's unique Id against query string parameter 'dioId'.

RequestContext

Once authroized to access the DigitalObject referenced by given AccessPath, a RequestContext object is set on HttpServletRequest scope. RequestContext allows access to only the permitted DigitalObjects. Except for posting comments RequestContext does not allow any modifications to application's database. RequestContext manages the persistence context whilst serving the content to the visitors.

Applications are supposed to the API on RequestContext to gain access to the DigitalObject the accessPath is pointing to, instead of instanciating a new PersistenceContext by creating the EntityManager.

ShareFilter is configured with the persistence unit name, and will create a new PersistenceContext (EntityManager) for every Http Request received. This entity manager instance is injected into the RequestContext object created.

This persistence context is closed when Http Response is committed.

Using ShareFilter for access control will mean that contacts accessing the content will have read-only access and cannot contribute changes to local database apart from posting a comment.

Dekoh Applications that intend to provide more flexibility as per the access levels to remote access of content are expected to implement their ways of access control.

LocalRequestContext

When ShareFilter determines that the request is made locally (not made over the internet), an instance of LocalRequestContext is set in the request scope. LocalRequestContext allows write access to the Dekoh application's Database. Using the API on LocalRequestContext create, read update, delete operatiosn can be performed on the DigitalObjects, Collections, Shares and other entities.

Creating the Share

<<insert picture here about filter allowing local user to create share>>

Contact

A Contact represents the contacts the owner would have added to his Dekoh address book. <<insert screen shot of contacts list, and how to add contact from Dekoh portal UI>>

Sharing the Application

An owner can use the Dekoh Desktop UI to share any of the installed Dekoh Application to his contacts.

<<Insert screen shot of Dekoh portal API that shares application>> Apart from this manual method, Dekoh Applications can choose programatically share the application to contacts when ever content is shared to them.

Sharing content

DioCollections containing DigitalObjects can be shared with Contacts. Or more Contacts can be added to the existing Share.

Other AccessControl Models

Apart from the above discussed access control model where contacts have read-only access to shared DigitalObjects, there may be other models which are more suitable to a application's business' logic. Imagine a Wiki application, where a page can be edited by a selected group of people, while it can only be viewed by another set of people.

A 'page' in Wiki application can be tagged, shared to, and commented on. Lets assume that the Page will be a DigitalObject in the Wiki application installed on Dekoh platform. In such a Wiki application, the owner when sharing the a Page DigitalObject would want to administer write access to certain selection of contacts and read only access to another set. Using the Access control model that Dekoh platform provides, this cannot be implemented.

To achive this Wiki application must implement its own HttpServletFilter which allows modification of content of a Page by managing the trasaction in Persistence Context.

Setting up Authenticated access

Dekoh applications installed on a computer may be used by more than one Dekoh user who has access to the same computer. In this case access to a Dekoh application must be permitted only after authentication.

A Dekoh application can restrict access to parts of application to only the authenticated Dekoh users by setting up the HttpServletFilter implementation - 'dekoh.portal.login.PortalLoginFilter' to intercept request matching any URL pattern. On interception PortalLoginFilter will direct user to login page for authentication if he has not logged in already.

Example: The following XML snippet from a web.xml sets up the PortalLoginFilter to intercept all requests.

    <filter>
        <filter-name>LoginFilter</filter-name>
 
<filter-class>dekoh.portal.login.PortalLoginFilter</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>LoginFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

Privacy for Local users

Shares created by one local user are not visible to other local users on the same computer. Consequently comments made on a share will be seen only by the user that created the share. However content imported to Dekoh by one user will be visible to other users as well. The LocalRequestContext instance set in the request scope by the ShareFilter for requests made by owner, allows retrieving of any DigitalObjects imported to Dekoh Database.

Dekoh applications intending override this behavior can supply their implementations of the LocalRequestContext class to the ShareFilter. This can be done by supplying the class name of the implementation of LocalRequestContext as a init-parameter in the declaration of ShareFilter in the web.xml.

<filter>
        <filter-name>Share Filter</filter-name>
        <filter-class>dekoh.portal.fwk.share.ShareFilter</filter-class>
        <init-param>
            <param-name>persistence-unit-name</param-name>
            <param-value>photo-dio-persistence-unit</param-value>
        </init-param>
        <init-param>
__            <param-name>lrc-impl-class</param-name>__
__            <param-value>__
__fully qulified class name of application's implementation of LocalRequestContext__
__   </param-value>__
        </init-param>
 </filter>

RSS

Using Dekoh platform API Dekoh applications can choose to publish the creation of Share as a RSS feed that can be subscribed by contacts. If contacts themselves have Dekoh installed on their computers this RSS feed is automatically aggregated. Similary Dekoh applications can publish RSS feed about a contact visiting the share, or posting a comment. This feed will be shown on the local user's Dekoh Desktop.

RSS for new Share created

Extend ShareChannel to provide implementations of that will pull from the application event stream a list of Shares created recently for a Contact.

An implementation of ShareChannelProvider should be registered with SyndicationService to publish this feed in various feed formats (RSS 1, RSS2).

RSS feed of Comments

Extend CommentChannel to provide implementations of that will pull most recent comments on shares created by the given local user. An implementation of CommentChannelProvider should be registered with SyndicationService to publish this feed in various feed formats (RSS 1, RSS2).

RSS feed of Contact visit

Not everyone of your contacts that visits the shared content will post a comment. So knowing that someone has visited will be indicative of the interest the shared content drawing in a user's contact network. Dekoh applications can record the contact's visit to view the share, by calling dekoh.portal.Stats.recordBuddyVisit(HttpServletRequest) in the relavant JSP.

Application should extend BuddyVisitChannel to provide implementations of that will supply access link and preview image for the visited share to be displayed in the RSS. An implementation of BuddyChannelProvider should be registered with SyndicationService to publish this feed in various feed formats (RSS 1, RSS2).

Import Service

To share the any content stored as files right off users desktops using Dekoh, these files have to be mapped to a DigitalObject instance by the Dekoh application. For example, Dekoh Photos application lets users import photos from their desktops to Dekoh Photos applications. These photos can then be shared with contacts.

Dekoh Music application imports mp3 files from desktop before letting the music to be shared. This process of creating a DigitalObjects representing files on disc is called importing. Importing files into Dekoh application usually involves the following steps.

1. Read metadata about the file. Metada is data apart from the actual data, usually adding contextual / or value added information. Some file formats like JPEG, MP3 earmark a blocks of the file to store metadata. JPEG files contain metadata like the make of the camera used, geographic location where the photo is taken etc. MP3 format includes metadata like duration of the song, the Genre of the song, the Artist and Album the song is part of.
2. Create a DigitalObject instance, set extracted metadata on the created instanace, the actual file location to Dekoh application's Database.
3. Organize the imported files into easy to find categories using the metadata gathered in step1. For example Dekoh Music application could organize the songs into DioCollections based on properties like Genre, Artist, Album. These properties can be read from the metadat of any MP3 file. Another way to categorize DigitalObject is to add Tags. Though Tags are usually picked by the users, Dekoh applications may choose to derive information from the file's metadata that can be added as Tags. JPEG format allows devices and software to write user chosen keywords as metadata. These keywords can be added as Tags on created instance of the PhotoDio. Organization of created DigitalObjects so users can easily find what they are looking for is key factor in determining the ease of use of the Dekoh application.

The first two steps are repeated for each file found in the directories the user is importing. If the number of files matching the datatype the Dekoh application is interested in is huge, then this process is both time consuming, memory intensive and involves lot of writes to application Database.

A well designed import process will

1. Provide the status of import process indicating how many files are read and how many are remaining. or by providing a percentage of completion.
2. Changes to previously imported folder should be consistently detected and applied to Dekoh application's database. The user may have added or deleted files from the directory previously imported.
3. While importing file to Dekoh application, the user's CPU usage, memory usage remain at optimal levels.

Dekoh platform provides API (Import Service API) and UI widgets that lets Dekoh Applications to import files of chosen type from directories into Dekoh application.

<<Insert screen shots of Photo app location chooser, import progress bar, and import complete>>

Dekoh platform's Import Service API provides progress of the import process by emitting events whenver a file with matching datatype is found and is imported to Dekoh application. A started import process can be stopped at any time. Dekoh's Import Service makes sure that only the changes made to a previously imported directory are imported.

During the import process many DigitalObjects are created. Each DigitalObject is a JPA entity. As the number of JPA entities created in a persistence context increase, the heap usage also increases. To keep the runtime memory in check Import Service API batches the persists to Dekoh application's Database, and clears the persistence context periodically. Hence by using Dekoh's Import Service a Dekoh application can perform a well behaved and monitorable import process.

How to use Import Service API

Dekoh platform's Import Service API, comprises of the following classes.

ImportConfig

An instance ImportConfig is a required input to start import process. The input will contain FilenameFilter implementation to obtain only the files that the Dekoh application wants handle. ImportConfig will also have the list of directories to be imported from.

ImportContext

An instance of ImportContext is required input to start import process. ImportConfig and application data directory are needed to create instance of this class. ImportService will create an import in the supplied application data directory.

ImportProcessor

Implementation of this interface will create DigitalObject for each file being imported and will manage the Transaction. The Persistence context (Transaction) with in which the DigitalObjects are created and presisted to Database is managed by methods beginImport() and commitChanges().

Dekoh platform API includes a partial implementation of ImportProcessor in AbstractImportProcessor. This abstract implementation will manage the Persistence context, and will require extending classes to implement only the methods to create DigitalObject from a file, find if a DigitalObject already exists for a given file.

   /**
     * Creates a DigitalObject instance representing this file.
     *
     * @param file
     * @return
     */
    protected DigitalObject createDigitalObject(File file)
    {        
    }

    /**
     * Find the DigitalObject if one already exists for the given file. If DigitalObject already exists, improt process
     * will skip processing this file.
     *
     * @param file
     * @param sourceDir directory entity for the parent dir of supplied file.
     * @return
     */
    protected DigitalObject findByFileAndSourceDir(File file, Directory
sourceDir)
    {        
    }

    /**
     * Gets the source File from which the supplied DigitalObject was created. Used to find the files on disk that are
     * removed, for which Digital objects still exist.
     *
     * @param digitalObject represents a file, because files from User's desktop are being imported.
     * @return file
     */
    protected File getSourceFile(DigitalObject digitalObject)
    {     
    }

Every created DigitalObject is added to the DioCollection representing the directory and is persisted to Database. This default categorization of DigitalObject according to the directories they are found in can be overridden by overriding method :

public ImportStatus importDigitalObject(File rootDir, File f)

More Customizations
Dekoh applications for which there is no one-to-one relationship between the file on the disc and the DigitalObject instance can implement ImportProcessor to gain more control over the import process.

Invoke the Import Service

• Create instance of the ImportService
• Call start() method. As a result of executing start() method, importing process will start in a new thread. start method return an instance of ImportProgressObject.
• The caller thread is expected to keep track of progress of import process by adding a listener on the ImportProgressObject.

dekoh.portal.fwk.importer.ImportProgressEventListenerImpl is a implementation of Progress listener provided by Dekoh. This implementation will provide percentage of completion and keeps a list of new, modified, deleted files as seen by the running import process.

The following pseudo code explains the above three steps.

ImportService importService = new ImportService(componentId); importService.registerProcessor(new MyImportProcessor()); ImportContext importContext = new ImportContext(new ImportConfig(newDirectoriesToImport, filenameFilter), rootDirForApp);rootDirForApp); importProgressObject = new ImportProgressObject(); importProgressObject = importService.start(importContext,
importProgressObject); importProgressObject.addImportProgressListener(listener);
...
for (; (!importProgressObject.getProgressStatus().isCompleted());) {  try {
   Thread.sleep(waitTime);
 } catch (InterruptedException e) {
   logger.logThrowable(e);
 }
 int percent = listener.getPercentageComplete();  System.out.println('Percentage complete :'+percent+' %');  
}

To simplify the programing model for ImportProcessor implementations, it is assumed that there will not more than one import service running at a given time per Dekoh Application.

To ensure that only one import Service instance is running at any given point in time, applications can set a global flag whenever Import is started. Every request to start new import will check if this flag is set. When the previously running import process finishes, this global flag can be cleared.

for (; (!importProgressObject.getProgressStatus().isCompleted());) 
{  try {
   Thread.sleep(waitTime);
 } catch (InterruptedException e) {
   logger.logThrowable(e);
 }
 int percent = listener.getPercentageComplete();  System.out.println('Percentage complete :'+percent+' %');  } //Importing is done, clear the flag now clearFlag(); 

Utiltiy class dekoh.portal.util.ExecuteImportHelper takes care of this by setting the instance of ImportProgressObject returned by the start() method in caller applications servletContext. When ever a request to start import is made from an application, this utility will check the servletContext of that application for a ImportProgressObject instance. If this instance exists and the method

importProgressObject.getProgressStatus().isCompleted() 

returns false, the request to start launch a new import is denied.

public static String __doImport__(ServletContext servletContext,
                             File[] newDirectoriesToImport,
                             FilenameFilter fileNameFilter,
                             String componentId,
                             ImportProcessor importProcessor,
                             Boolean blockUntilCompletion,
                             ImportProgressEventListener
importProgressEventListener)

Parameters

• servletContext - servlet context of the web application that is calling this method
• newDirectoriesToImport - list of directories to import from. will recurse the sub directories.
• fileNameFilter - implementation to filter from the list of files in a dir to arrive at the files the calling application is interested in
• componentId - component id of the application that is calling this method

importProcessor -

• blockUntilCompletion - when set to true caller thread will block until completion of the import
• importProgressEventListener - this listener instance will be registerd with the ImportProgressObject of the launched import

Painless Application Upgrades

As any application evolves, new versions of the application will be released. Usually newer versions carry a promise of more exciting new features and fixes to issues reported against previous releases. However preserving backwards compatibility with previous releases is of paramount importance.

Users of previous releases may have accumilated valuable data, that should continue to work with newer versions. In other words the application upgrade should be seamless to end users, harness the familiarity they would have gained from using the previous version of application.

Application installation and upgradation is managed by Dekoh. Dekoh runtime emits events during the first time installation, and application upgrade. Applications can register a listener to these events by specifying its fully qualified class name in the component.xml Example:

<component>  <!-- snipped for brevity-->
<callback-classes>
<callback-class>dekoh.portal.install.PortalAppsUpdateHelper</callback-cl
ass>
</callback-classes>    
</component>

These callback class must implement com.pramati.bfly.vm.update.UpdateHandler .

    public void preInstall(InstallEnv env) throws InstallationException
    {
    }

    public void postInstall(InstallEnv env) throws InstallationException
    {
    }

    public void preUninstall(InstallEnv env) throws InstallationException
    {
    }

    public void postUninstall(InstallEnv env) throws InstallationException
    {
    }

For the implementation of UpdateHanlder to be picked up by Dekoh runtime, this class and other resources it depends on are expected to be packaged to a jar named 'callback.jar'. This jar file should be avialable at the root of the applicationWAR file.

Upgrading Application Database

Often times because of new features added to the application, the database schema changes. In such cases when the application is being upgraded, its database also has to be migrated to conform to the new schema. Applications should enable users to upgrade the latest version of application from all previously released versions. The database migration may involve many changes which should either happen all at once or not at all. An installation log reporting any failures will also help recovering the database from unusable state. Dekoh 'PortalAppsUpdateHelper' offers help in managing the changes to application database schema.

PortalAppsUpdateHelper reads the current version of the application, knows the new version to upgrade to. Applications using this class are expected to provide sql files containing SQL statements that will migrate the application database.These SQL files must be named according to following rule. <oldVersion>_<newVersion>_migration.sql

For example, to migrate application database from version 1 to 2 the migration SQL file should be named:

1_2_migration.sql

To migrate application database from version 1 to 2 the migration SQL files should be named:

1_2_migration.sql
2_3_migration.sql

PortalAppsUpdateHelper will execute the SQL statements in 1_2_migration.sql and then execute the statmetns in 2_3_migration.sql.

How to use PortalAppsUpdateHelper

Instead of implementing all the methods in com.pramati.bfly.vm.update.UpdateHandler to take care of database migration, dekoh application can choose to extend dekoh.portal.install.PortalAppsUpdateHelper.

Not every application's release requires Database migration. So applications can maintain a different version number.