Use WebDAV with WebSphere Portal

Use WebDAV with WebSphere Portal

Overview
Alternatives to WebDAV
Task webdav-deploy-zip-file
Enable write access for WebDAV file store
Use WebDAV with the portal theme
Folder structure and reserved folder names
Theme folders
Other folders
Filestore cache control
Supported HTTP methods
Serve HTTP OPTIONS requests to the server context root by WebDAV clients
*.tmp files
Appendix

Overview

We can leverage Web-based Distributed Authoring and Versioning (WebDAV) to allow editing of web content using Windows tools. To use configure a WebDAV client, for example CyberDuck, AnyClient or BitKinex.
I have had success recently with WinSCP

After set up, we can access web content libraries using the following URL:
http://server:port/portal_context_root/mycontenthandler/dav/content/libraries/

For example
http://localhost:10039/wps/mycontenthandler/dav/content/libraries/

To get to folders...
http://server:port/portal_context_root/mycontenthandler/dav/fs-type1

For example...
http://localhost:10039/wps/mycontenthandler/dav/fs-type1

As an alternative to using WebDAV for theme files, can incorporate the theme files into the theme EAR.
We can create components or presentation templates by dragging a file into a corresponding folder. We can perform actions on several items at once. For example, drag multiple images into the image component folder. For each image, the file name is used for the component's name and title.
We can modify modify XML files defining the item's metadata and access control characteristics.
Using your preferred HTML editor, such as vi, we can open...

libraries
taxonomies
categories
site areas
folders
components
presentation templates

The WebDAV client accesses the web content library, downloads the template, and then uploads the changes.
WebDAV does not support editing...

Content items
Authoring templates
Nested items within site areas
Server-side copy and move
Unauthenticated users
Exporting of web content libraries with WebDAV to be imported into another web content server

Locking or unlocking an item through WebDAV will lock or unlock the item in Web Content Manager and the JCR database. Locking an item, folders, and files related to the content of the item locks the item's metadata and access control settings.
The file must be in a workflow state that allows editing.
Files representing data items are always named exactly like the corresponding content item. For example if you have an image component named myImage, the corresponding image file is also named myImage, without any suffix indicating the file type, such as .gif or .jpg. This can sometimes cause a problem when opening the file through WebDAV because the appropriate application for editing the file cannot be chosen automatically. To work around, either rename the component itself to include the file type (for example, myImage.gif), or manually start the editing application, and open the file from within the client.
The WebDAV tree begins at the WebDAV root...
/libraries/

Examples of WebDAV services...

WebDAV for managing pages and static content.
WebDAV filestore for portal themes.
WebDAV for WCM.

WebDAV does not support https.
To use a Web server with WebDAV:

From the WAS admin console select...
Web servers | webserver name | Plug-in properties | Request and response

...and set...
Accept content for all requests = true

Regenerate the web server plug-in.

Copy the file plugin-cfg.xml to the Plugin directory.

Open plugin-cfg.xml and set AcceptAllContent to true.

Restart the web server.

Alternatives to WebDAV

Rather than use WebDAV to store and retrieve static content, we can package static content for themes as part of the theme ear file. After expanding the ear, there are two subfolders, one for dynamic content, and one for static content. For example...
MyCoThemeDynamic-0.0.1-SNAPSHOT.war
MyCoThemeStatic-0.0.1-SNAPSHOT.war

Task webdav-deploy-zip-file
To manage Page Builder theme artifacts, and to deploy iWidgets, use the webdav-deploy-zip-file task to upload archive or compressed files to portal WebDAV folders.
To create the archive file to deploy, on the source host, create a zip file of theme with all your static resources organized by type in to folders.
When you execute webdav-deploy-zip-file, by default the referenced target folder is replaced by the extracted contents of the referenced archive or compressed file. If there are files or folders contained in the referenced WebDAV folder, those files and folders are deleted before the new content is added. To avoid this behavior and merge the contents, set -DUpdateMode=merge to merge the contents of the archive into the content that already exists at the target URI.
If the WebDAV folder referenced by the target URI does not exist yet, it is created; in this case make sure to have a trailing slash ( / ) at the end of the target URI.
This task uploads archive or compressed files to portal WebDAV folders.
Use this task to upload archive or compressed files to portal WebDAV folders. For example, we can upload the archive MyModuleTheme.zip to the public folder of the file store by doing either one of the following:
Invocation example...
cd /opt/IBM/portal/wp_profile/ConfigEngine
./ConfigEngine.sh webdav-deploy-zip-file \
                  -DTargetURI=dav:fs-type1/themes/MyTheme/ \
                  -DZipFilePath=/path/to/MyModularTheme.zip \
                  -DUpdateMode=merge 
Mandatary parameters to be specified on the command line or in the file wkplc.properties...

WasUserid WebSphere Application Server user ID
WasPassword WAS password
PortalAdminId Portal administrator user ID
PortalAdminPwd Portal administrator password
TargetURI URI of the WebDAV folder where we want the archive or compressed file to be extracted. In the example, it resolves to:
PROFILE_HOME/temp/nodename/WebSphere_Portal/JCRFileStore/filestore/fs-type1/themes/MyTheme
ZipFilePath File system path to the archive or compressed file. Do not use this parameter combined with the parameter ZipFileClassPath.
ZipFileClassPath Java class path to the to the archive or compressed file. Do not use this parameter combined with the parameter ZipFilePath.
UpdateMode (=replace) Default value for this parameter is replace. To merge the content of the archive or compressed file with the content that already exists at the given URI, set this parameter to the value merge. In this case file that do not yet exist are created, existing files are updated, and no files are deleted.
VirtualPortalContext
VirtualPortalHost Identify the virtual portal. Only pages contained in the specified virtual portal are refreshed. If you omit this parameter, by default no virtual portal page layout is refreshed.

An alternative to using webdav-deploy-zip-file is to copy MyModuleTheme.zip to the portal class path, for example...
AppServer/lib/ext

Enable write access for WebDAV file store

By default only administrative users can perform write operations on specific folders of the WebDAV file store.
To enable write access for all authenticated users, in the WAS admin console, for the WP ConfigService resource environment provider, set...
filestore.writeaccess.allowed = true

Restart the portal server for the new setting to take effect.
Setting true applies to the filestore entry points for home directories for each user located at:
http://server:WC_default_host/wps/mycontenthandler/dav/fs-type1/users/user_name

Users can modify only their own directories.

Use WebDAV with the portal theme

Entry point URL for themes...
http://www.myco.com:10039/wps/mycontenthandler/dav/fs-type1/

Theme resources...
http://myco.com:10039/wps/mycontenthandler/dav/fs-type1/themes

Skin resources...
http://myco.com:10039/wps/mycontenthandler/dav/fs-type1/skins

Identify virtual portals using either host name or URL context...
http://vp.mycompany.com:10039/wps/mycontenthandler/dav/fs-type1/themes
http://localhost:10039/wps/mycontenthandler/vp1/!ut/p/dav/fs-type1/themes

We can obtain the entry point URL to the WebDAV file store from the service document under the URL...
/wps/mycontenthandler/!ut/p/model/service-document

The service document contains the top-level access point as follows:
<app:collection href="http://setgetweb.com/p/portal80//webdav/!ut/p/dav/fs-type1/">

     <atom:title>fs-type1</atom:title>

     <app:categories fixed="yes">
          . . . . .            
          <atom:category term="webdav"/>
          <atom:category term="filestore"/>
          . . . . .       
     </app:categories>

</app:collection>
Folder structure and reserved folder names

The fs-type1 WebDAV entry points provide the following set of predefined root folders used by themes:

themes
skins
layout-templates
common-resources
iwidgets

The fs-type1 WebDAV entry points also provide the following set of predefined root folders. We can access them using the Remote Model function provided by the Enabler API:

public
users

The fs-type1 WebDAV entry points provide the following internal folder:

system

None of the folders listed here can be deleted, not even by an administrator.
The two sets of folders differ in the access control policy that guards access to the resources contained in those folders.

Theme folders

Each folder represents a WebDAV collection. Administer write access to the theme folders via the virtual resource THEME MANAGEMENT provided by portal access control.

/themes Store resources associated to themes, such as theme templates. Typically, each subfolder represents one theme.
/skins Global skins. Typically, each subfolder of this folder represents one global skin.
/layout-templates Templates for layouts that can be used by individual themes. Typically, each subfolder represents one layout template.
/common-resources Share common resources between different themes, so that they can be managed in a single place.
/iwidgets Widgets

WebDAV prevents the deletion of these folders. Even users with administrator rights cannot delete these folders and the data in them.

All users have view access to all resources in these folders. This includes both anonymous users and authenticated users.
To give users write access to resources contained in these folders, for example to create, modify, or delete such resources, assign the users MANAGER role on the virtual resource THEME MANAGEMENT in portal access control.

Other folders

Each of these folders represents a WebDAV collection. The access control policy for them is hardcoded as described for each folder.
/public All authenticated users have read and write access to this folder. Anonymous users have read access only.
/users All authenticated users have read access only to this folder. Anonymous users have read access only.
/users/user_name
Only the user user_name has access to these files. Created for the individual user user_name when the user accesses the WebDAV file store for the first time. The portal uses the user IDs of the individual users as the names for the users' folders user_name. internally the portal uses the VMM ID of the user, so that the data does not need to be moved when the user name is changed.
To programmatically find out the URL entry point to such folder for the current user we can look into the services document for the following entry...
<app:collection href="http://setgetweb.com/p/portal80//webdav/!ut/p/dav/fs-type1/users/<username>">
     <atom:title>fs-type1-user</atom:title>
     <app:categories fixed="yes">
          . . . . .
          <atom:category term="webdav"/>
          <atom:category term="filestore"/>
          <atom:category term="user"/>
          . . . . .          
     </app:categories>
</app:collection>
/users/user_name/public The user user_name has read and write access to this folder. This folder contains content that the user user_name shared with other users. Portal access control mapping: inherited. All authenticated users have read access to this folder. Anonymous users have read access to this folder. All other subfolders of /users/user_name can only the accessed by the user user_name.
/system The system folder is reserved for system internal information. Administrators can view this folder in WebDAV. Other portal users cannot view this folder. Portal access control mapping: None.
Filestore cache control

The WebDAV file store supports serving timeout values for HTTP Cache Header entries.
We can use regular expressions to specify the time out value for elements in the filestore folder structure that match the regular expression. You need to add the following two custom properties to the WP Config Service resource environment provider with the following key = value properties:
filestore.cache.expiration.id.re=regular expression filestore.cache.expiration.id.seconds=value

The id value can consist of an arbitrary string. It is used only to establish the mapping between a regular expression and its associated timeout value. If there are multiple regular expressione that match a given filestore resource, the maximum of the associated time out values will be used.
All items under the /themes folder have an expiration time of 1800 seconds:
filestore.cache.expiration.0.re=themes/.* filestore.cache.expiration.0.seconds=1800

All items of a certain resource type, such as jpg or gif have an expiration time of 6000 seconds:
filestore.cache.expiration.1.re=.*\.jpg|.*\.gif filestore.cache.expiration.1.seconds=6000

All css files in the themes folder have an expiration time of 8000 seconds:
filestore.cache.expiration.2.re=themes/.*\.css filestore.cache.expiration.2.seconds=8000

Supported HTTP methods

WebDAV file store supports the following HTTP methods:

Method Description
PROPFIND Get properties, including collection structure, resource names, sizes, and dates of last modification
MKCOL Create new folders
GET Retrieve resources. Requires View role access rights
HEAD Retrieve HTTP headers of resources. Requires View role access rights
POST Upload new resources
DELETE Delete resources or folders
PUT Update resources, such as documents or images
COPY Copy resources or folders
MOVE Move or rename resources or folders

Serve HTTP OPTIONS requests to the server context root by WebDAV clients

Some WebDAV clients send an HTTP OPTIONS request to the server context root ( / ) to check whether the server supports WebDAV. To support these clients, the portal provides a web application called wp.webdav.options.was.
To enable this WAR file, adapt the application.xml files of the deployed Enterprise Application (EAR), wps.ear, so that wp.webdav.options.war is mapped to the context root ( / )...
Export the EAR, wps.ear, through the WAS admin console.
Expand the EAR:

./EARExpander.sh -ear directory/WebDAV_for_WebSphere_Portal.ear -operationDir directory webdav_expanded -operation expand
Edit application.xml in the expanded EAR file directory and uncomment or add the following section:
<module>
     <web>
          <web-uri>wp.webdav.options.war</web-uri>
          <context-root>/</context-root>
     </web>
</module>
Collapse the EAR:
./EARExpander.sh -ear directory/wps.ear  \
                 -operationDir directory/wps_expanded  \
                 -operation collapse \
Update the enterprise application with these changes using the WAS admin console.

Save the changes.

Restart the portal for the changes to take effect.
Depending on how the web server is set up, this change might cause all requests to be routed to the web server plug-in. In this case refer to the web server documentation for information about how to route only OPTIONS requests to the plug-in.

*.tmp files

Recently, during a cumulative fix upgrade to a portal v8 instance, CONFIG-WP-PTF-CF failed with an OutOfMemoryError...
[zip] Building zip: C:\IBM\WebSphere\wp_profile\ConfigEngine\config\work\themedavupdates\updateddavfiles.zip
Target finished: action-update-fp-wp.theme.themes/default80-files
--- Exception Thrown ---
C:\IBM\WebSphere\PortalServer\theme\wp.theme.themes\default80\config\includes\wp.theme.themes.default80.fp_cfg.xml:38: java.lang.OutOfMemoryError: Java heap space

The zip file referenced in the exception represents a collection of theme files stored in the Portal JCR database. In our case it turned out that thousands of *.tmp files had accumulated in WebDAV. So many in fact, that the upgrade did not have enough heap space to process them all. IBM support indicated that this behavior is rare, and not an issue on all Portal v80x environments.
To diagnose...

Installed BitKinex WebDAV client.
Configured BitKinex to connect to primary node.
Http/WebDAV (right-click) | New | Http/WebDAV

...and enter server address:
localhost.mydomain:10039

user id + password is main Portal administrator account
After connecting, right click and change directory to...
/myapp/mycontenthandler/dav/fs-type1/

If nothing appears to be happening, click the Refresh button.

Delete every .tmp file in this directory

Continue by traversing through every sub-folder. We found *.tmp files under each hone.
Portal8.0\contributions\
Portal8.0\css\
etc...

Because upgrade itself seems to be creating the *.tmp files, we tried updating Ant task to skip the *.tmp files when performing zip, using the procedure below. But that did not work either.
Locate and backup the following file:
copy C:\IBM\WebSphere\PortalServer\theme\wp.theme.themes\default80\config\includes\wp.theme.themes.default80.fp_cfg.xml c:\temp
Edit the file ...
notepad C:\IBM\WebSphere\PortalServer\theme\wp.theme.themes\default80\config\includes\wp.theme.themes.default80.fp_cfg.xml

...and change...
<zip destfile="${work.dir}/themedavupdates/updateddavfiles.zip"
     basedir="${work.dir}/themedavupdates/davfiles">
    <include name="themes/Portal8.0/**/*"/>
</zip>
<zip destfile="${work.dir}/themedavupdates/updateddavfiles.zip"
     basedir="${work.dir}/themedavupdates/davfiles">
    <include name="themes/Portal8.0/**/*"/>
    <exclude name="**/*.tmp" />
</zip>
Save changes. Retry the upgrade.
Final outcome pending. I'll keep you posted.

Appendix

Caching Proxy supports WebDAV methods used by Microsoft Exchange Server, and user-defined (customized) methods. These methods are hard coded and managed by the Enable and Disable directives. Administrators can also use the corresponding method-mask defined in the PROTECT directive to authorize the use of these methods.
WebDAV for IBM WebSphere Portal provides a way to administer portal resources. Both administrators and users can use it. WebDAV for WebSphere Portal allows users to administer portal pages and content of static pages of a portal using standard operating system tools. This way client side administrators and users can browse, read, and write these resources using file explorers or editors.
Portal pages are represented as folders. They can contain subfolders representing child pages. Static pages are a special case: the content of a static page is located in a separate folder named staticcontent under the main folder on that page.
Users can work in the folders as usual, for example by performing drag-and-drop operations. The folders also hold property file containing metadata for portal resources, such as the title and description. Users can edit the property files to update portal resources. When the user saves the updated file, the updates are transferred and applied to the portal directly.
All the files shown in WebDAV are virtual files created from the page data in the portal database

We can use WebDAV to

Browse through the page hierarchy. Each portal page is represented as a folder. The name of the folder is the unique name or the object ID of the page. Children pages in the topology are represented as subfolders.

Change globalization information of pages. To do this, users edit and save properties file containing the globalization information.

Change metadata of pages. To do this, users edit and save properties file containing the metadata information.

Delete pages.

For static pages only, we can browse, read, create, update, save, copy, move, and delete static content, such as HTML or image files. Users can access the content of static pages via the subfolder staticcontent of the page.

We cannot use WebDAV to perform any of the following tasks:

Create new pages.
Update portal content.
Modify the unique name or objectID of pages.
Move or copy pages.

WebSphere Portal contains the WebDAV service and enablement layer. Before using WebDAV for WebSphere Portal, users must set up their WebDAV client.
After setting up the WebDAV client, they can connect to WebDAV for WebSphere Portal and work with portal pages and content. To connect to WebDAV for WebSphere Portal, they enter the WebDAV entry URL.
The WebDAV entry point requires user authentication via HTTP basic authentication. SSL access is not supported at this time. To use WebDAV, users log in to the portal with their portal user ID. Users can access and work with portal pages according to their access permissions as set by Portal Access Control.
The WebDAV entry URL is as follows:

For default portal installations:
http://server:port/wps/mycontenthandler/dav/contentmodel/wps.content.root/

For virtual portals:

If a host name was specified when the virtual portal was created, the WebDAV URL looks like this:
http://vphost:port/wps/mycontenthandler/dav/contentmodel/wps.content.root/

If the virtual portal was created with a URL context only and no host name was specified, the WebDAV URL looks like this:
http://server:port/wps/mycontenthandler/vpcontext/!ut/p/dav/contentmodel/wps.content.root/

To connect to WebDAV to work with portal pages and static content, you enter the WebDAV entry URL.
The WebDAV entry URL is as follows:

For default portal installations:
http://server:port/wps/mycontenthandler/dav/contentmodel/page_unique_name/

...or...
http://server:port/wps/mycontenthandler/dav/contentmodel/page_object_id/

For virtual portals:

If a host name was specified when the virtual portal was created, the WebDAV URL is as follows:
http://vphost:port/wps/mycontenthandler/dav/contentmodel/page_unique_name/

...or...
http://vphost:port/wps/mycontenthandler/dav/contentmodel/page_object_id/

If the virtual portal was created with a URL context only and no host name was specified, the WebDAV URL is as follows:
http://server:port/wps/mycontenthandler/vpcontext/!ut/p/dav/contentmodel/page_unique_name/

...or...
http://server:port/wps/mycontenthandler/vpcontext/!ut/p/dav/contentmodel/page_object_id/

Where:

server Host name of the portal server.
port Port number of WebSphere Portal.
vpcontext URL context of the target virtual portal, if the virtual portal URL context is configured to be encoded into the URL.
vphost Host name of the target virtual portal, if the virtual portal host name is configured to be encoded into the URL.
page_unique_name Unique name for the portal page.
page_object_id objectID for the portal page as it appears in the Manage Pages portlet.

Examples of entry URLs to all portal pages...
http://www.myco.com:10039/wps/mycontenthandler/dav/contentmodel/wps.content.root/

For virtual portals examples of entry URLs to all portal pages are as follows:

For a virtual portal created with the host name vp.mycompany.com:
http://vp.mycompany.com:10039/wps/mycontenthandler/dav/contentmodel/wps.content.root/

For a virtual portal created with the URL context vp1 and without a host name:
http://localhost:10039/wps/mycontenthandler/vp1/!ut/p/dav/contentmodel/wps.content.root/

Related:

Virtual portals

Parent: Administering

WasUserid	WebSphere Application Server user ID
WasPassword	WAS password
PortalAdminId	Portal administrator user ID
PortalAdminPwd	Portal administrator password
TargetURI	URI of the WebDAV folder where we want the archive or compressed file to be extracted. In the example, it resolves to: PROFILE_HOME/temp/nodename/WebSphere_Portal/JCRFileStore/filestore/fs-type1/themes/MyTheme
ZipFilePath	File system path to the archive or compressed file. Do not use this parameter combined with the parameter ZipFileClassPath.
ZipFileClassPath	Java class path to the to the archive or compressed file. Do not use this parameter combined with the parameter ZipFilePath.
UpdateMode (=replace)	Default value for this parameter is replace. To merge the content of the archive or compressed file with the content that already exists at the given URI, set this parameter to the value merge. In this case file that do not yet exist are created, existing files are updated, and no files are deleted.
VirtualPortalContext VirtualPortalHost	Identify the virtual portal. Only pages contained in the specified virtual portal are refreshed. If you omit this parameter, by default no virtual portal page layout is refreshed.

/themes	Store resources associated to themes, such as theme templates. Typically, each subfolder represents one theme.
/skins	Global skins. Typically, each subfolder of this folder represents one global skin.
/layout-templates	Templates for layouts that can be used by individual themes. Typically, each subfolder represents one layout template.
/common-resources	Share common resources between different themes, so that they can be managed in a single place.
/iwidgets	Widgets

/public	All authenticated users have read and write access to this folder. Anonymous users have read access only.
/users	All authenticated users have read access only to this folder. Anonymous users have read access only.
/users/user_name	Only the user user_name has access to these files. Created for the individual user user_name when the user accesses the WebDAV file store for the first time. The portal uses the user IDs of the individual users as the names for the users' folders user_name. internally the portal uses the VMM ID of the user, so that the data does not need to be moved when the user name is changed. To programmatically find out the URL entry point to such folder for the current user we can look into the services document for the following entry... <app:collection href="http://setgetweb.com/p/portal80//webdav/!ut/p/dav/fs-type1/users/<username>"> <atom:title>fs-type1-user</atom:title> <app:categories fixed="yes"> . . . . . <atom:category term="webdav"/> <atom:category term="filestore"/> <atom:category term="user"/> . . . . . </app:categories> </app:collection>
/users/user_name/public	The user user_name has read and write access to this folder. This folder contains content that the user user_name shared with other users. Portal access control mapping: inherited. All authenticated users have read access to this folder. Anonymous users have read access to this folder. All other subfolders of /users/user_name can only the accessed by the user user_name.
/system	The system folder is reserved for system internal information. Administrators can view this folder in WebDAV. Other portal users cannot view this folder. Portal access control mapping: None.

Method	Description
PROPFIND	Get properties, including collection structure, resource names, sizes, and dates of last modification
MKCOL	Create new folders
GET	Retrieve resources. Requires View role access rights
HEAD	Retrieve HTTP headers of resources. Requires View role access rights
POST	Upload new resources
DELETE	Delete resources or folders
PUT	Update resources, such as documents or images
COPY	Copy resources or folders
MOVE	Move or rename resources or folders

server	Host name of the portal server.
port	Port number of WebSphere Portal.
vpcontext	URL context of the target virtual portal, if the virtual portal URL context is configured to be encoded into the URL.
vphost	Host name of the target virtual portal, if the virtual portal host name is configured to be encoded into the URL.
page_unique_name	Unique name for the portal page.
page_object_id	objectID for the portal page as it appears in the Manage Pages portlet.