Problem Statement:
The article addresses the problem of making REST-based calls from AEM to an external system and the best way to handle HTTP requests.
Requirement:
The article discusses the implementation of an OSGi-based REST service in AEM that integrates with an external system using the HTTP Client factory. The author provides detailed steps on how to create a new Apache Closable HTTP Client, prepare request configuration, pool HTTP connections, and use default headers and keepAlive strategy to execute requests.
Create an OSGi based REST service to integrate AEM with the external system, and also provide config to provide endpoint options and client factory configurations.
Introduction:
As we all know AEM is REST based Web application, however, is there a way to integrate OSGi based service to make calls to the external system.
After going through the ACS commons based HTTP Client factory, I created a more feature-friendly and rich HTTP client factory.
Create HTTPClientFactory Service Interface:
This service provides the implementation for most of the basic HTTP REST based operations like GET, PUT, POST, and DELETE operations.
package com.example.core.services;
import org.apache.http.client.fluent.Executor;
import org.apache.http.client.fluent.Request;
/**
* Factory for building pre-configured HttpClient Fluent Executor and Request objects
* based a configure host, port and (optionally) username/password.
* Factories will generally be accessed by service lookup using the factory.name property.
*/
public interface HttpClientFactory {
/**
* Get the configured Executor object from this factory.
*
* @return an Executor object
*/
Executor getExecutor();
/**
* Create a GET request using the base hostname and port defined in the factory configuration.
*
* @param partialUrl the portion of the URL after the port (and slash)
*
* @return a fluent Request object
*/
Request get(String partialUrl);
/**
* Create a PUT request using the base hostname and port defined in the factory configuration.
*
* @param partialUrl the portion of the URL after the port (and slash)
*
* @return a fluent Request object
*/
Request put(String partialUrl);
/**
* Create a POST request using the base hostname and port defined in the factory configuration.
*
* @param partialUrl the portion of the URL after the port (and slash)
*
* @return a fluent Request object
*/
Request post(String partialUrl);
/**
* Create a DELETE request using the base hostname and port defined in the factory configuration.
*
* @param partialUrl the portion of the URL after the port (and slash)
*
* @return a fluent Request object
*/
Request delete(String partialUrl);
/**
* Create a OPTIONS request using the base hostname and port defined in the factory configuration.
*
* @param partialUrl the portion of the URL after the port (and slash)
*
* @return a fluent Request object
*/
Request options(String partialUrl);
/**
* Get External URI type is form the factory configuration.
*
* @return External URI Type
*/
String getExternalURIType();
/**
* Get apiStoreLocatorHostName URI type is form the factory configuration.
*
* @return API StoreLocatorHost
*/
String getApiStoreLocatorHostName();
Request postWithAbsolute(String absolutelUrl);
}
Create HTTPClientFactoryConfig:
Add the required attributes to create the HTTPCLientFactory.
package com.example.services.config;
import org.osgi.service.metatype.annotations.AttributeDefinition;
import org.osgi.service.metatype.annotations.AttributeType;
import org.osgi.service.metatype.annotations.ObjectClassDefinition;
import com.example.constants.Constants;
@ObjectClassDefinition(name = "Http Client API Configuration", description = "Http Client API Configuration")
public @interface HttpClientFactoryConfig {
@AttributeDefinition(name = "API Host Name", description = "API host name, e.g. https://example.com", type = AttributeType.STRING)
String apiHostName() default Constants.DEFAULT_API_HOST_NAME;
@AttributeDefinition(name = "API URI Type Path", description = "API URI type path, e.g. /services/int/v2", type = AttributeType.STRING)
String uriType() default Constants.DEFAULT_API_URI_TYPE_PATH;
@AttributeDefinition(name = "API URI Type Path", description = "API URI type path, e.g. /services/ext/v2", type = AttributeType.STRING)
String uriExternalType() default Constants.DEFAULT_API_URI_EXTERNAL_TYPE_PATH;
@AttributeDefinition(name = "Relaxed SSL", description = "Defines if self-certified certificates should be allowed to SSL transport", type = AttributeType.BOOLEAN)
boolean relaxedSSL() default Constants.DEFAULT_RELAXED_SSL;
@AttributeDefinition(name = "Store Locator API Host Name", description = "Store Locator API host name, e.g. https://example.com", type = AttributeType.STRING)
String apiStoreLocatorHostName() default Constants.DEFAULT_STORE_LOCATOR_API_HOST_NAME;
@AttributeDefinition(name = "Maximum number of total open connections", description = "Set maximum number of total open connections, default 5", type = AttributeType.INTEGER)
int maxTotalOpenConnections() default Constants.DEFAULT_MAXIMUM_TOTAL_OPEN_CONNECTION;
@AttributeDefinition(name = "Maximum number of concurrent connections per route", description = "Set the maximum number of concurrent connections per route, default 5", type = AttributeType.INTEGER)
int maxConcurrentConnectionPerRoute() default Constants.DEFAULT_MAXIMUM_CONCURRENT_CONNECTION_PER_ROUTE;
@AttributeDefinition(name = "Default Keep alive connection in seconds", description = "Default Keep alive connection in seconds, default value is 1", type = AttributeType.LONG)
int defaultKeepAliveconnection() default Constants.DEFAULT_KEEP_ALIVE_CONNECTION;
@AttributeDefinition(name = "Default connection timeout in seconds", description = "Default connection timout in seconds, default value is 30", type = AttributeType.LONG)
long defaultConnectionTimeout() default Constants.DEFAULT_CONNECTION_TIMEOUT;
@AttributeDefinition(name = "Default socket timeout in seconds", description = "Default socket timeout in seconds, default value is 30", type = AttributeType.LONG)
long defaultSocketTimeout() default Constants.DEFAULT_SOCKET_TIMEOUT;
@AttributeDefinition(name = "Default connection request timeout in seconds", description = "Default connection request timeout in seconds, default value is 30", type = AttributeType.LONG)
long defaultConnectionRequestTimeout() default Constants.DEFAULT_CONNECTION_REQUEST_TIMEOUT;
}
Create HttpClientFactoryImpl Service implementation:
This provides the implementation class for HTTPClientFactory Service and during @Activate/@Modified we are trying to create a new Apache Closable HTTP Client using OSGi based HttpClientBuilderFactory.
HTTP client is like a dish, and you can taste it better if your recipe is great and if you prepare it well, before making calls to the external system.
Close all Connections:
Make sure to close the existing connection if any after bundle gets activated or modified
Preparing Request Configuration:
Create Request Config Object and set Connection timeout, socket timeout, and request timeout based on the service configurations
Pooling HTTP Connection:
PoolingHttpClientConnectionManager maintains a pool of HttpClientConnections and is able to service connection requests from multiple execution threads. Connections are pooled on a per route basis. A request for a route that already the manager has persistent connections for available in the pool will be serviced by leasing a connection from the pool rather than creating a brand new connection.
Hence set the max pool size and number default max per route (per endpoint)
Things to be aware of before pooling connection is, are you making HTTPS calls to the external system if yes? Then create an SSLConnectionSocketFactory with NOOP based verifier and add all the trusted certificates.
Default Keep Alive Strategy:
If the Keep-Alive header is not present in the response, HttpClient assumes the connection can be kept alive indefinitely. However, many HTTP servers in general use are configured to drop persistent connections after a certain period of inactivity to conserve system resources, often without informing the client. In case the default strategy turns out to be too optimistic, one may want to provide a custom keep-alive strategy.
HTTP Client Builder OSGi Service:
Get the reference to OSGi-based httpClientBuilderFactory service, prepare a new builder, set the request configuration, and add a connection manager with a pooling connection.
Add default headers and keepAlive strategy, so that we don’t have to create a new connection
Finally, create the HTTP Client out of this builder and set the client to Apache fluent Executor.
the fluent executor makes an arbitrary HttpClient instance and executes the request.
package com.example.core.services.impl;
import java.io.IOException;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.Header;
import org.apache.http.HttpResponse;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.fluent.Executor;
import org.apache.http.client.fluent.Request;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.ConnectionKeepAliveStrategy;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustAllStrategy;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.message.BasicHeader;
import org.apache.http.osgi.services.HttpClientBuilderFactory;
import org.apache.http.protocol.HttpContext;
import org.apache.http.ssl.SSLContextBuilder;
import org.osgi.service.component.annotations.Activate;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Deactivate;
import org.osgi.service.component.annotations.Modified;
import org.osgi.service.component.annotations.Reference;
import org.osgi.service.metatype.annotations.Designate;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.example.core.services.HttpClientFactory;
import com.example.core.services.config.HttpClientFactoryConfig;
/**
* Implementation of @{@link HttpClientFactory}.
* <p>
* HttpClientFactory provides service to handle API connection and executor.
*/
@Component(service = HttpClientFactory.class)
@Designate(ocd = HttpClientFactoryConfig.class)
public class HttpClientFactoryImpl implements HttpClientFactory {
private static final Logger log = LoggerFactory.getLogger(HttpClientFactoryImpl.class);
private Executor executor;
private String baseUrl;
private String uriExternalType;
private String apiStoreLocatorHostName;
private CloseableHttpClient httpClient;
private HttpClientFactoryConfig config;
@Reference
private HttpClientBuilderFactory httpClientBuilderFactory;
@Activate
@Modified
protected void activate(HttpClientFactoryConfig config) throws Exception {
log.info("########### OSGi Configs Start ###############");
log.info("API Host Name : {}", config.apiHostName());
log.info("URI Type: {}", config.uriType());
log.info("########### OSGi Configs End ###############");
closeHttpConnection();
this.config = config;
if (this.config.apiHostName() == null) {
log.debug("Configuration is not valid. Both hostname is mandatory.");
throw new IllegalArgumentException("Configuration is not valid. Both hostname is mandatory.");
}
this.uriExternalType = this.config.uriExternalType();
this.apiStoreLocatorHostName = this.config.apiStoreLocatorHostName();
this.baseUrl = StringUtils.join(this.config.apiHostName(), config.uriType());
initExecutor();
}
private void initExecutor() throws Exception {
PoolingHttpClientConnectionManager connMgr = null;
RequestConfig requestConfig = initRequestConfig();
HttpClientBuilder builder = httpClientBuilderFactory.newBuilder();
builder.setDefaultRequestConfig(requestConfig);
if (config.relaxedSSL()) {
connMgr = initPoolingConnectionManagerWithRelaxedSSL();
} else {
connMgr = new PoolingHttpClientConnectionManager();
}
connMgr.closeExpiredConnections();
connMgr.setMaxTotal(config.maxTotalOpenConnections());
connMgr.setDefaultMaxPerRoute(config.maxConcurrentConnectionPerRoute());
builder.setConnectionManager(connMgr);
List<Header> headers = new ArrayList<>();
headers.add(new BasicHeader("Accept", "application/json"));
builder.setDefaultHeaders(headers);
builder.setKeepAliveStrategy(keepAliveStratey);
httpClient = builder.build();
executor = Executor.newInstance(httpClient);
}
private PoolingHttpClientConnectionManager initPoolingConnectionManagerWithRelaxedSSL()
throws NoSuchAlgorithmException, KeyStoreException, KeyManagementException {
PoolingHttpClientConnectionManager connMgr;
SSLContextBuilder sslbuilder = new SSLContextBuilder();
sslbuilder.loadTrustMaterial(new TrustAllStrategy());
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslbuilder.build(),
NoopHostnameVerifier.INSTANCE);
Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory()).register("https", sslsf).build();
connMgr = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
return connMgr;
}
private RequestConfig initRequestConfig() {
return RequestConfig.custom()
.setConnectTimeout(Math.toIntExact(TimeUnit.SECONDS.toMillis(config.defaultConnectionTimeout())))
.setSocketTimeout(Math.toIntExact(TimeUnit.SECONDS.toMillis(config.defaultSocketTimeout())))
.setConnectionRequestTimeout(
Math.toIntExact(TimeUnit.SECONDS.toMillis(config.defaultConnectionRequestTimeout())))
.build();
}
@Deactivate
protected void deactivate() {
closeHttpConnection();
}
private void closeHttpConnection() {
if (null != httpClient) {
try {
httpClient.close();
} catch (final IOException exception) {
log.debug("IOException while clossing API, {}", exception.getMessage());
}
}
}
@Override
public Executor getExecutor() {
return executor;
}
@Override
public Request get(String partialUrl) {
String url = baseUrl + partialUrl;
return Request.Get(url);
}
@Override
public Request post(String partialUrl) {
String url = baseUrl + partialUrl;
return Request.Post(url);
}
@Override
public Request postWithAbsolute(String absolutelUrl) {
return Request.Post(absolutelUrl);
}
@Override
public Request put(String partialUrl) {
String url = baseUrl + partialUrl;
return Request.Put(url);
}
@Override
public Request delete(String partialUrl) {
String url = baseUrl + partialUrl;
return Request.Delete(url);
}
@Override
public Request options(String partialUrl) {
String url = baseUrl + partialUrl;
return Request.Options(url);
}
@Override
public String getExternalURIType() {
return uriExternalType;
}
@Override
public String getApiStoreLocatorHostName() {
return apiStoreLocatorHostName;
}
ConnectionKeepAliveStrategy keepAliveStratey = new ConnectionKeepAliveStrategy() {
@Override
public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
/*
* HeaderElementIterator headerElementIterator = new BasicHeaderElementIterator(
* response.headerIterator(HTTP.CONN_KEEP_ALIVE));
*
* while (headerElementIterator.hasNext()) { HeaderElement headerElement =
* headerElementIterator.nextElement(); String param = headerElement.getName();
* String value = headerElement.getValue(); if (value != null &&
* param.equalsIgnoreCase("timeout")) { return
* TimeUnit.SECONDS.toMillis(Long.parseLong(value)); } }
*/
return TimeUnit.SECONDS.toMillis(config.defaultKeepAliveconnection());
}
};
}
References:
https://github.com/kiransg89/AEM-REST-Integration
How to use HTTP Client Factory?
Check this out URI
AEM best practices and development guide 