Import large files and images

Because importing large files into Web Content Manager can have a negative impact on performance, we can adjust several settings to ensure better performance when importing files.

Before updating the settings for large file handling, review the following considerations:

UNIX note: If we are running on a UNIX operating system, ensure that you have used the ulimit -f command to set the maximum size of files that can be created to be at least the size of the largest file we would need to upload to the content server. The command ulimit -f unlimited removes any limit on file size. When setting the size, also make sure the system has sufficient disk space to support the setting.

Disk space requirements: When importing files, a temporary directory is used to store the files during the upload process. If the size of the uploaded files exceeds the available disk space for the temporary directory, the import operation fails. When uploading large files, ensure that there is sufficient disk space to accommodate the import. The location of the temporary directory is specified by the jcr.binaryValueFileDir property in the WP_PROFILE/PortalServer/jcr/lib/com/ibm/icm/icm.properties file.

1. Log in to the WebSphere Integrated Solutions Console.

2. Click Resources > Resource Environment > Resource Environment Providers > WCM WCMConfigService > Custom properties.

   Cluster note: If we are using this web content server as part of a cluster, ensure that we use the WebSphere Integrated Solutions Console for the deployment manager when manipulating configuration properties.

3. For the resource.maxUploadSize property, specify a value in megabytes corresponding to the size of the largest file to allow to be imported. For example, if you do not want to allow files larger than 34 MB to be imported, update the resource.maxUploadSize property to have a value of 34. Although IBM recommends that this value not exceed 100 MB, we cannot upload files larger than 512 MB.

4. For the resourceserver.maxCacheObjectSize property, specify a value of 300 KB or less.

5. Add the transaction.sync.remove property, and specify a value of true.

6. Click Servers > Server Types > WebSphere appservers > portal_server > Server infrastructure > Administration > Custom Properties

7. Add the protocol_http_large_data_inbound_buffer property, and for the value specify the maximum file size in bytes. This value should correspond to the value you set for the resource.maxUploadSize property in the WCM WCMConfigService service.

   Note that the protocol_http_large_data_inbound_buffer property uses bytes. So if you specified a value of 34 MB for the resource.maxUploadSize property, we would specify a value of 35651584 bytes for the protocol_http_large_data_inbound_buffer property.

8. Click Resources > JDBC > Data sources > datasource_name > Custom properties

9. Specify the fullyMaterializeLobData property with a value of false.

10. Click Resource > JDBC > Data sources > datasource_name > Connection pool properties.

11. Increase the maximum number of database collections allowed for the application server by increasing the value of the Maximum connections field to a value greater than the default 50 connections.

12. If we are working with files larger than 100 MB, increase the web containers transaction timeout setting.

    1. Click Servers > Server Types > WebSphere appservers > portal_server > Container Services > Transaction service.

    2. Increase the value of the Total transaction lifetime timeout setting from the default setting of 120 seconds.

13. Increase the maximum number of threads allowed in the thread pool used by the web container.

    1. Click Servers > Server Types > WebSphere appservers > portal_server > Thread pools > WebContainer.

    2. Set the value of the Maximum Size field to 100 threads.

14. For IBM HTTP Server Version 7, increase the connection timeout value for connections to the application.

    1. Click Servers > Server Types > web servers > web_server > plug-in properties > Custom properties > New.

    2. In the name field, enter ServerIOTimeout.

    3. In the value field, enter the timeout value in seconds.

       The default value is 60 seconds. However, when working with large files, this default value is typically insufficient and can cause a false server error response to be sent, which in turn causes the portal to reissue the request. Specify a timeout value that is long enough to allow a failing request to receive a response, or enter -1 for an unlimited timeout value.

15. Click Save to save the configuration changes.

16. Restart the portal for the settings to take effect.

If the portal's policy cache manager indicates that a number of web container threads are hung, set the cacheinstance.com.ibm.wps.policy.services.PolicyCacheManager.lifetime property in the WP CacheManagerService service to a value of -1. This setting reduces the database connections and load times and helps prevent threads from hanging.

Parent: Configure a web content authoring environment

Related:

Set service configuration properties