ColdFusion (2016 release) Update 14

ColdFusion (2016 release) Update 14 (release date, 17 March, 2020) includes fixes for the the security vulnerabilities that were reported in APSB20-16.

A configuration issue with AJP Protocol was fixed in this update which affects ColdFusion 2016 and 2018 along with a few JEE application servers, which use AJP such as Tomcat, JBoss, and Wildfly.

ColdFusion uses Tomcat as application server and uses AJP connector to allow web servers, such as, IIS and Apache to use AJP Protocol connector for communication. AJP is a highly trusted protocol and must never be exposed to untrusted clients. It is insecure (clear text transmission) and assumes that your network is safe. If you are using a web server or connector configuration set up, you must make sure that the Port and IP address on which the AJP is running is only available to the web server.

If you are not using any web server and are using an in-built web server configuration, you can completely comment out or remove the AJP connector from the server.xml configuration.

Before this update, the AJP connector could bind all the interfaces (0.0.0.0) on the machine on which ColdFusion runs. From this update, the AJP connector will bind to localhost address (127.0.0.1 or ::1, depends on the machine) by default, if the address attribute is not specified.

When you are applying this update, we add a random secret to the server.xml AJP connector unless it is already configured. Also, Tomcat requires that the AJP connector must be configured with a secret otherwise the AJP connector won't start (unless secretRequired=false is added to the AJP connector in server.xml).

Prerequisites

1. On 64-bit computer, use 32-bit JRE for 32-bit ColdFusion and 64-bit JRE for 64-bit ColdFusion.
2. If the ColdFusion server is behind a proxy, specify the proxy settings for the server to get the update notification and download the updates. Specify proxy settings using the system properties below in the jvm.config for a stand-alone installation, or corresponding script file for JEE installation.
   • http.proxyHost
   • http.proxyPort
   • http.proxyUser
   • http.proxyPassword
3. For ColdFusion running on JEE application servers, stop all application server instances before installing the update.

Installation

For instructions on how to install this update, see Server Update section. For any questions related to updates, see this FAQ. 

• The update can be installed from the Administrator of a ColdFusion instance or through the command-line option.
• Windows users can launch the ColdFusion Administrator using Start > All Programs > Adobe > Coldfusion 2016 > Administrator.
• Microsoft Windows 7, Windows 8, Windows 10, Windows Server 2008, and Windows Server 2012 users must use the “Run as Administrator” option to launch  wsconfig  tool at {cf_install_home}/{instance_name}/runtime/bin.
• If you get the following error when installing the update using the Download and Install option, ensure that the folder {cf_install_home}/{instance_name}/hf_updates has write permission: "An error occurred when performing a file operation write on file {cf_install_home}/{instance_name}/hf-updates/hotfix_014.properties".
• The connector configuration files are backed up at {cf_install_home}/config/ wsconfig /backup. Add back any custom changes made to the workers.properties file after reconfiguring the connector.

Installing the update manually

1. Click the link to download the JAR.
2. Execute the following command on the downloaded JAR. You must have privileges to start or stop ColdFusion service and full access to the ColdFusion root directory.

    Windows: <cf_root>/jre/bin/java.exe -jar <jar-file-dir>/hotfix-014-318307.jar

    Linux-based platforms: <cf_root>/jre/bin/java -jar <jar-file-dir>/hotfix-014-318307.jar

Ensure that the JRE bundled with ColdFusion is used for executing the downloaded JAR. For standalone ColdFusion, this must be at, <cf_root>/jre/bin.

Install the update from a user account that has permissions to restart ColdFusion services and other configured webservers .

For further details on how to manually update the application, see the help article.

Post installation

After applying this update, the ColdFusion build number should be 2016,0,14,318307.

Location of workers.properties and server.xml files

Locating server.xml

server.xml file contains configuration related to Tomcat and is located at {cf.instance.home}\runtime\conf directory.

Locating AJP Connector in server.xml

AJP connector has protocol attribute set as AJP/1.3. For example,

<Connector port="8018" protocol="AJP/1.3" packetSize="65535" redirectPort="8451" tomcatAuthentication="false" maxThreads="500" connectionTimeout="60000"/>

workers.properties is the file where you configure the secret so that the server can communicate with Tomcat.

Locating workers.properties in IIS

The workers.properties file can be found at {cf.home}/config/wsconfig/<magic_folder_number>\workers.properties.

Locating workers.properties in Apache

The workers.properties file can be found at APACHE_HOME/conf/workers.properties.

After applying the update, depending on your configuration, follow these steps.

Web server (Connector) and ColdFusion are on the same machine

After applying the update:

If you have already locked down ColdFusion, then you need not take any action, since ColdFusion instances are already configured with the requiredSecret attribute and your web server also has the secret information with it to communicate. You can reverify the same by looking at server.xml and workers.properties (worker.<INSTANCE_NAME>.secret=YOUR_AJP_SECRET) files. 

If you have not locked down ColdFusion, then upgrade all your ColdFusion instances. The ColdFusion update configures AJP connector in all these ColdFusion instances with a random secret.

Run the wsconfig utility, which lists down all the configured connectors. Select each connector and click on the upgrade connector. Clicking on the upgrade connector copies the secret from the instances and puts it in the workers.properties file. Restart IIS or Apache and then test your websites. 

If you face any network-related issues, refer to the section Troubleshooting below.

Web server and ColdFusion instance(s) are running on different machines

After updating the ColdFusion instance, ColdFusion configures AJP connector in the instance with a random secret.

Copy the secret from instance to workers.properties. For example,

worker.cfusion.secret=<secret>

If you have multiple instances, after updating all the instances, copy secret from each instance to worker.sproperties. For example,

worker.<instance>.secret=<secret_from_instance>

Before installing the update, ColdFusion instance ran on 0.0.0.0 (all interfaces), but now it binds to localhost loopback address. Therefore, the remote web server is unable to communicate with Tomcat.

Configure each ColdFusion instance with Tomcat IP address so that Tomcat can be accessed over remote connector. Restart ColdFusion instances. Also, change your firewall configuration such that only trusted hosts such as web server will be allowed to access the AJP connector.

<Connector port="8018" protocol="AJP/1.3" packetSize="65535" redirectPort="8451" tomcatAuthentication="false" maxThreads="500" connectionTimeout="60000" address="YOUR_TOMCAT_IP_ADDRESS"/>

Using built-in web server

If you are not using any external web server such as Apache/IIS and just using Tomcat http connector, comment out or remove the AJP connector from server.xml as it is not getting used. For example, comment out the following line in server.xml:

<Connector port="8018" protocol="AJP/1.3" packetSize="65535" redirectPort="8451" tomcatAuthentication="false" maxThreads="500" connectionTimeout="60000"/>

Using JEE application server

If you are using any of the specified application servers, you should upgrade your JEE container and follows the mitigation steps. The steps are more less same as mentioned above for those application servers.

Boss EAP or Wildfly Application servers:

https://access.redhat.com/solutions/4851251

Tomcat servers:

Upgrade to the latest Tomcat release according to the version of Tomcat release you are using. Refer to Apache Tomcat documentation.

WebLogic and WebSphere servers:

We recommend you to refer to the official documentation for more details.

Troubleshooting

ERROR 503

Locating server.xml

server.xml file contains configuration related to Tomcat and is located at {cf.instance.home}\runtime\conf directory.

Locating AJP Connector in server.xml

AJP connector has protocol attribute set as AJP/1.3. For example,

<Connector port="8018" protocol="AJP/1.3" packetSize="65535" redirectPort="8451" tomcatAuthentication="false" maxThreads="500" connectionTimeout="60000"/>

ColdFusion and web server are not on the same machine

Follow the steps below:

1. Ensure that Apache Tomcat AJP connector starts without any issue. Check ColdFusion logs for below messages:

    Mar 16, 2020 6:27:39 PM org.apache.coyote.AbstractProtocol start

    INFO: Starting ProtocolHandler ["ajp-nio-127.0.0.1-8014"]

OR

Check whether AJP port is open using a command like netstat.

    Windows: netstat -ano | findstr <AJP_PORT>

    Linux: netstat -apn | grep <AJP_PORT>

2. Check whether the web server and Tomcat can communicate.

ColdFusion and web server are on the same machine

Based on your /etc/hosts setup, Tomcat can bind to either 127.0.0.1 (IPV4 ) or ::1 (IPv6) or any configured address. You can check this by looking at the coldfusion-out.log file.

If the AJP connector binds to 127.0.0.1., the log records:

    Mar 16, 2020 6:27:39 PM org.apache.coyote.AbstractProtocol start

    INFO: Starting ProtocolHandler ["ajp-nio-127.0.0.1-8014"]

If the AJP connector binds to to 0:0:0:0:0:0:0:1 (IPv6), the log records:

    Mar 16, 2020 6:27:39 PM org.apache.coyote.AbstractProtocol start

    INFO: Starting ProtocolHandler ["ajp-nio-0:0:0:0:0:0:0:1-8014"]

The network issue between web server and Tomcat may occur if Tomcat runs on, for example, IPv4 loopback (127.0.0.1) and the web server tries to connect using IPv6 loopback address. 

If you want to run Tomcat on IPv6 loopback address, add address attribute to the AJP connector in server.xml, shown below, and restart the ColdFusion instance.

<Connector protocol="AJP/1.3" address="::1" port="8014" redirectPort="8447" packetSize="65535" secret="a3cbab43-dbe0-4372-90c9-ef5ddf2e84cf" tomcatAuthentication="false"/>

Alternatively, you can also change the address using which web server uses to connect to the Tomcat instances. This is useful if you have a cluster with number of instances in the same machine instead of changing the address attribute to each and every instance in server.xml.

You can change the address directly from workers.properties (worker.cfusion.host=127.0.0.1) using which web server uses to connect to all instances.

ERROR 403

Follow the steps below:

1. If you have not already upgraded the connectors, upgrade the connectors in all the instances using the command wsconfig -upgrade.
2. If you are still seeing 403 errors even after having a matching secret values, there might be a possibility that your web server or its connector might be setting additional request attributes which Tomcat does not allow. Some of the built-in request attributes required for SSL communication were already taken care by Tomcat. Add allowedRequestAttributesPattern=".*" to the AJP connector tag and restart the ColdFusion instance and see if you are able to access the website. If it works, you need to find out those request attributes and add it to attribute as a RegEx. Please refer to Tomcat AJP connector documentation for more information.

Uninstallation

To uninstall the update, perform one of the following:

• In ColdFusion Administrator, click Uninstall in Server Update > Updates > Installed Updates.
• Run the uninstaller for the update from the command prompt. For example, java -jar {cf_install_home}/{instance_home}/hf_updates/hf-2016-00014-318307/uninstall/uninstaller.jar

If you can't uninstall the update using the above-mentioned uninstall options, the uninstaller could be corrupted. However, you can manually uninstall the update by doing the following:

1. Delete the update jar from {cf_install_home}/{instance_name}/lib/updates.
2. Copy all folders from {cf_install_home}/{instance_name}/hf-updates/{hf-2016-00014-318307}/backup directory to {cf_install_home}/{instance_name}/

Connector configuration