Retry Utils in AEM using Java Streams

Problem Statement:

  1. How to retry a connection?
  2. Retry posting the data at least some time until the response is 200
  3. Retry any OAK operations, when the exception occurs

Introduction:

Usually, with respect to AEM, we don’t have Retry Utils which can retry the particular operation whenever an exception occurred.

If we are doing multiple transactions on the AEM repository, especially on a particular node like updating properties or updating references, the OAK operation would through exceptions like your operation is blocked by another operation or invalid modification.

If we are connecting to external services through REST API and the connection has failed or timeout and if we want to connect to the external system then we don’t have the option to retry out of the box

Create a Retry Utils as shown below:

Retry on Exception:

package com.test.utils;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryOnException {
    private static final Logger log = LoggerFactory.getLogger(RetryOnException.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int counter = 0; counter < maxTimes; counter++) {
            try {
                call.process();
                return true;
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (counter + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

The above Util can be used in any code as shown below and the retry will happen only when the exception occurs during operations

package com.test.utils;

import java.util.concurrent.atomic.AtomicInteger;
import org.aarp.www.mcp.utils.RetryOnException;

public class ExampleOne {
	public static void main(String[] args) {
		AtomicInteger atomicCounter = new AtomicInteger(0);
		try {
			RetryOnException.withRetry(3, 500, () -> {
				if(atomicCounter.getAndIncrement() < 2) {
					System.out.println("Retring count with Exception" + atomicCounter.get());
					throw new Exception("Throwing New Exception to test");
				} else {
					System.out.println("Retring count without Exception " + atomicCounter.get());
				}

			});
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}
Exception Result

Retry on the condition:

package com.test.utils;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryOnCondition {
    private static final Logger log = LoggerFactory.getLogger(RetryOnCondition.class);

    public static interface CallToRetry {
        boolean process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int counter = 0; counter < maxTimes; counter++) {
            try {
                boolean status = call.process();
                if(status) {
                	return true;
                }                
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (counter + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

The above Util can be used to retry based on the condition like if the connection is successful or the response code is 200

package com.test.utils;

import java.util.concurrent.atomic.AtomicInteger;
import org.aarp.www.mcp.utils.RetryOnCondition;

public class ExampleTwo {
	public static void main(String[] args) {
		AtomicInteger atomicCounter = new AtomicInteger(0);
		try {
			RetryOnCondition.withRetry(3, 500, () -> onCondition(atomicCounter));
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

	private static boolean onCondition(AtomicInteger atomicCounter) throws Exception {		
		if(atomicCounter.getAndIncrement() < 2) {
			System.out.println("Retring count with Condition false " + atomicCounter.get());
			return false;
		} else {
			System.out.println("Retring count with Condition true " + atomicCounter.get());
			return true;
		}
	}
}
On Condition result

Retry n number of times:

package org.aarp.www.mcp.utils;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryNTimes {
    private static final Logger log = LoggerFactory.getLogger(RetryNTimes.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int counter = 0; counter < maxTimes; counter++) {
            try {
                call.process();
                if(counter == (maxTimes-1)) {
                	return true;
                }                
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (counter + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

The above Util can be used in operations like revalidating the results, for example, any query results can be validated.

package com.test.utils;

import java.util.concurrent.atomic.AtomicInteger;
import org.aarp.www.mcp.utils.RetryNTimes;

public class ExampleThree {
	public static void main(String[] args) {
		AtomicInteger atomicCounter = new AtomicInteger(0);
		try {
			RetryNTimes.withRetry(3, 500, () -> {
				System.out.println("Retrying 3 times count " + atomicCounter.getAndIncrement());
			});
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}
Retying umtil retry count reached result

Perfect Content backup Package in AEM

Problem statement:

Create a perfect package in AEM, whenever we want to create any package in AEM we provide the content paths in the package filter we get only content pages. But what about images and reference pages? What about experience fragment pages or XF page-related context hub variations?

Introduction:

MCP (Manage Controlled Processes) is both a dashboard for performing complex tasks and a rich API for defining these tasks as process definitions. In addition to kicking off new processes, users can also monitor running tasks, retrieve information about completed tasks, halt work, and so on.

Add the following maven dependency to your pom to extend MCP

<dependency>
    <groupId>com.adobe.acs</groupId>
    <artifactId>acs-aem-commons-bundle</artifactId>
    <version>5.0.4</version>
    <scope>provided</scope>
</dependency>

In order to create a perfect content package in AEM, please create an MCP as shown below:

Create Process Definition factory – PackageCreatorFactory

This class tells ACS Commons MCP to pick the process definition and process name getName and you need to mention the implementation class inside the createProcessDefinitionInstance method as shown below:

package com.mysite.mcp.process;

import org.apache.jackrabbit.vault.packaging.Packaging;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import com.adobe.acs.commons.mcp.ProcessDefinitionFactory;
import com.adobe.acs.commons.packaging.PackageHelper;

@Component(service = ProcessDefinitionFactory.class, immediate = true)
public class PackageCreatorFactory extends ProcessDefinitionFactory<PackageCreator> {
	
	@Reference
    private PackageHelper packageHelper;
	
	@Reference
    private Packaging packaging;
	
    @Override
    public String getName() {
        return "Package Creator";
    }

    @Override
    protected PackageCreator createProcessDefinitionInstance() {
        return new PackageCreator(packageHelper, packaging);
    }
}

Create Process Definition implementation – PackageCreator

This is an implementation class where we are defining all the form fields required for the process to run

package com.mysite.mcp.process;

import com.adobe.acs.commons.fam.ActionManager;
import com.adobe.acs.commons.mcp.ProcessDefinition;
import com.adobe.acs.commons.mcp.ProcessInstance;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.TextareaComponent;
import com.adobe.acs.commons.mcp.model.GenericReport;
import com.adobe.acs.commons.mcp.model.ManagedProcess;
import com.adobe.acs.commons.packaging.PackageHelper;
import com.mysite.mcp.utils.ReportRetryUtils;
import com.mysite.mcp.visitors.ContentVisitor;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.packaging.*;
import org.apache.sling.api.resource.LoginException;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.jetbrains.annotations.NotNull;

import javax.jcr.RepositoryException;
import javax.jcr.Session;
import java.io.IOException;
import java.util.*;
import java.util.stream.Collectors;
import java.util.stream.Stream;

/**
 * Package Creator to create fully functional package for the given path
 */
public class PackageCreator extends ProcessDefinition {

	private final GenericReport report = new GenericReport();	
	
	private static final String DEFUALT_GROUP_NAME = "my_packages";
	private static final String DEFUALT_VERSION = "1.0";
	private static final String QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH = "/apps/acs-commons/components/utilities/packager/query-packager/definition/package-thumbnail.png";
	private static final String REPORT_NAME = "Package_Creator";
    
    @FormField(name = "Page Paths",
            description = "Comma-separated list of components to exclude",
            required = false,
            component = TextareaComponent.class,
            options = {"default=/content/"})
    private String pagePaths;
    
    @FormField(name = "Package Name",
            description = "Custom package name",
            required = false,
            options = {"default=backing up"})
    private String packageName;
    
    @FormField(name = "Package Description",
            description = "Custom package Description",
            required = false,
            options = {"default=Package created by package creator"})
    private String packageDescription;

    ManagedProcess instanceInfo;
    private PackageHelper packageHelper;
    private Packaging packaging;

    public PackageCreator(PackageHelper packageHelper, Packaging packaging) {
    	this.packageHelper = packageHelper;
    	this.packaging = packaging;
	}

	@Override
    public void init() throws RepositoryException {
    }

    @Override
    public void buildProcess(ProcessInstance instance, ResourceResolver resourceResolver) throws LoginException, RepositoryException {
    	instanceInfo = instance.getInfo();
    	instance.getInfo().setDescription("Generating package");
    	instance.defineAction("Preparing package", resourceResolver, this::createPackage); 
    	report.setName(REPORT_NAME);
    }
    
    protected void createPackage(ActionManager manager) {
    	manager.deferredWithResolver(this::collectReferences); 
    }
    
    private void collectReferences(ResourceResolver resourceResolver) throws Exception {
    	Set<String> contentPaths = new HashSet<>();
    	Set<String> xfPaths = new HashSet<>();
    	Set<String> damPaths = new HashSet<>();    	
    	ContentVisitor contentVisitor =  new ContentVisitor();
    	Set<String> pages = new HashSet<>(Arrays.asList(pagePaths.split(","))).stream().map(r -> r + "/jcr:content").collect(Collectors.toSet());
    	iterateContentPaths(resourceResolver, contentPaths, xfPaths, damPaths, contentVisitor, pages);
    	damPaths.addAll(iterateContent(resourceResolver, contentVisitor, damPaths));
    	ReportRetryUtils.withRetry(2, 300, () -> xfPaths.addAll(iterateContent(resourceResolver, contentVisitor, xfPaths)));    	    	
    	Set<String> allPaths = Stream.of(pages, contentPaths, xfPaths, damPaths).flatMap(Collection::stream).collect(Collectors.toSet());
        packagePaths(resourceResolver, allPaths);
    }

	private Set<String> iterateContent(ResourceResolver resourceResolver, ContentVisitor contentVisitor, Set<String> pages) {
		Set<String> interimPaths = new HashSet<>();
		for(String page : pages) {
			contentVisitor.accept(resourceResolver.resolve(page));
			interimPaths.addAll(contentVisitor.getContentPaths());
			interimPaths.addAll(contentVisitor.getDamPaths());
			interimPaths.addAll(contentVisitor.getXfPaths());
		}
		return interimPaths;
	}

	private void iterateContentPaths(ResourceResolver resourceResolver, Set<String> contentPaths, Set<String> xfPaths,
			Set<String> damPaths, ContentVisitor contentVisitor, Set<String> pages) {
			for(String page : pages) {
				contentVisitor.accept(resourceResolver.resolve(page));
				contentPaths.addAll(contentVisitor.getContentPaths());
				xfPaths.addAll(contentVisitor.getXfPaths());
				damPaths.addAll(contentVisitor.getDamPaths());
			}
	}

	private void packagePaths(ResourceResolver resourceResolver, Set<String> allPaths)
			throws IOException, RepositoryException, PackageException {
		Map<String, String> packageDefinitionProperties = new HashMap<>();
        // ACL Handling
        packageDefinitionProperties.put(JcrPackageDefinition.PN_AC_HANDLING,
        		AccessControlHandling.OVERWRITE.toString());

        // Package Description
        packageDefinitionProperties.put(JcrPackageDefinition.PN_DESCRIPTION, packageDescription);
    	Set<@NotNull Resource> packageResources = allPaths.stream().map(resourceResolver::resolve).collect(Collectors.toSet());
    	try(JcrPackage jcrPackage = packageHelper.createPackage(packageResources,
				resourceResolver.adaptTo(Session.class), DEFUALT_GROUP_NAME, packageName, DEFUALT_VERSION,
				PackageHelper.ConflictResolution.Replace, packageDefinitionProperties)){
    		// Add thumbnail to the package definition
            packageHelper.addThumbnail(jcrPackage, resourceResolver.getResource(QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH));
            final JcrPackageManager packageManager = packaging.getPackageManager(resourceResolver.adaptTo(Session.class));
            packageManager.assemble(jcrPackage, null);
            recordAction(jcrPackage.getNode().getPath(), "Built", "Package Built is successful");
    	}		
	}


    public enum ReportColumns {
        PATH, ACTION, DESCRIPTION
    }

    List<EnumMap<ReportColumns, String>> reportData = Collections.synchronizedList(new ArrayList<>());

    private void recordAction(String path, String action, String description) {
        EnumMap<ReportColumns, String> row = new EnumMap<>(ReportColumns.class);
        row.put(ReportColumns.PATH, path);
        row.put(ReportColumns.ACTION, action);
        row.put(ReportColumns.DESCRIPTION, description);
        reportData.add(row);
    }

    @Override
    public void storeReport(ProcessInstance instance, ResourceResolver rr) throws RepositoryException, PersistenceException {    	
        report.setRows(reportData, ReportColumns.class);
        report.persist(rr, instance.getPath() + "/jcr:content/report");
    }
}

Add the Utils called ReportRetryUtils as shown below:

package com.mysite.mcp.utils;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class ReportRetryUtils {
    private static final Logger log = LoggerFactory.getLogger(ReportRetryUtils.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int counter = 0; counter < maxTimes; counter++) {
            try {
                call.process();
                if(counter == (maxTimes-1)) {
                	return true;
                }                
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (i + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

Add the class called ContentVistor as shown below:

package com.mysite.mcp.visitors;

import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.resource.*;
import org.jetbrains.annotations.NotNull;
import java.util.*;
import java.util.regex.Pattern;

public class ContentVisitor extends AbstractResourceVisitor {
	
	private static final String CONTENT_SLASH = "/content/";
	private static final String CONTENT_DAM_SLASH = "/content/dam/";
	private static final String CONTENT_XF_SLASH = "/content/experience-fragments/";
	
	Pattern htmlPattern = Pattern.compile(".*\\<[^>]+>.*", Pattern.DOTALL);
	
	Set<String> contentPaths = new HashSet<>();
	Set<String> xfPaths = new HashSet<>();
	Set<String> damPaths = new HashSet<>();
	
	@Override
	public final void accept(final Resource resource) {
		if (null != resource && !ResourceUtil.isNonExistingResource(resource)) {
			final ValueMap properties = resource.adaptTo(ValueMap.class);
			final String primaryType = properties.get(ResourceResolver.PROPERTY_RESOURCE_TYPE, StringUtils.EMPTY);
			if(StringUtils.isNoneEmpty(primaryType) || StringUtils.startsWith(resource.getPath(), "/content/dam/content-fragments")){
				visit(resource);
			}
			this.traverseChildren(resource.listChildren());
		}
	}

	@Override
	protected void traverseChildren(final @NotNull Iterator<Resource> children) {
		while (children.hasNext()) {
			final Resource child = children.next();
			accept(child);
		}
	}

	@Override
	protected void visit(final @NotNull Resource resource) {
		resource.getValueMap().entrySet().forEach(property -> {
			Object prop = property.getValue();
			if (prop.getClass() == String[].class) {
				List<String> propertyValue = Arrays.asList((String[]) prop);
				if (!propertyValue.isEmpty()) {							
					propertyValue.stream().filter(s -> StringUtils.isNotEmpty(s) && StringUtils.startsWith(s, CONTENT_SLASH) && !htmlPattern.matcher(s).matches()).forEach(this::populateValue);
				}
			} else if (prop.getClass() == String.class) {
				String propertyValue = (String) prop;
				if (StringUtils.isNotEmpty(propertyValue) && StringUtils.startsWith(propertyValue, CONTENT_SLASH) && !htmlPattern.matcher(propertyValue).matches()) {
					populateValue(propertyValue);
				}
			}
		});
	}

	private void populateValue(String value) {
		if(StringUtils.startsWith(value, CONTENT_DAM_SLASH)) {
			damPaths.add(value);
		} else if(StringUtils.startsWith(value, CONTENT_XF_SLASH)) {
			xfPaths.add(value+"/jcr:content");
		} else {
			contentPaths.add(StringUtils.substringBeforeLast(value, ".html")+"/jcr:content");		
		}
	}

	public Set<String> getContentPaths() {
		return contentPaths;
	}

	public Set<String> getXfPaths() {
		return xfPaths;
	}

	public Set<String> getDamPaths() {
		return damPaths;
	}	
}

Once the code is deployed, please go to the following URL and click on start process as shown below:

http://domain/apps/acs-commons/content/manage-controlled-processes.html

Start MCP Process

You will see a new process called Property Update as shown below:

Click on the process

Package Creator Process

Provide all the page paths as a comma separated values, package name, and descriptions before starting the process

Create Package Process Definition

Once the package is created in the results you can find the package location as shown below:

Create Package reutlt

As can see in the filter, it has packaged the content path along with related images and header and footer XF paths as well

Backup package in package manager view

Access CRX Package Manager in PROD

Problem statement:

How to access the CRX package manager in PROD or AEM as Cloud services?

User cases:

  1. Latest content package from PROD to lowers or local for debugging purposes
  2. Install the content package on PROD
  3. Continue the PROD deployment during CM outage in between deployment

Introduction:

Packages enable the importing and exporting of repository content. For example, you can use packages to install new functionality, transfer content between instances, and back up repository content.

A package is a zip file holding repository content in the form of a file-system serialization (called “vault” serialization). This provides an easy-to-use-and-edit representation of files and folders.

Packages include content, both page content and project-related content, selected using filters.

A package also contains vault meta information, including the filter definitions and import configuration information. Additional content properties (that are not used for package extraction) can be included in the package, such as a description, a visual image, or an icon; these properties are for the content package consumer and for informational purposes only.

In order to access packages in AEM:

MCP (Manage Controlled Processes) is both a dashboard for performing complex tasks and a rich API for defining these tasks as process definitions. In addition to kicking off new processes, users can also monitor running tasks, retrieve information about completed tasks, halt work, and so on.

Add the following maven dependency to your pom to extend MCP

<dependency>
    <groupId>com.adobe.acs</groupId>
    <artifactId>acs-aem-commons-bundle</artifactId>
    <version>5.0.4</version>
    <scope>provided</scope>
</dependency>

Create Process Definition factory – PackageHandlerFactory

This class tells ACS Commons MCP to pick the process definition and process name getName and you need to mention the implementation class inside the createProcessDefinitionInstance method as shown below:

package com.mysite.mcp.process;

import org.apache.jackrabbit.vault.packaging.Packaging;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import com.adobe.acs.commons.mcp.ProcessDefinitionFactory;

@Component(service = ProcessDefinitionFactory.class, immediate = true)
public class PackageHandlerFactory extends ProcessDefinitionFactory<PackageHandler> {
	
    @Reference
    private Packaging packaging;
	
    @Override
    public String getName() {
        return "Package Handler";
    }

    @Override
    protected PackageHandler createProcessDefinitionInstance() {
        return new PackageHandler(packaging);
    }
}

Create Process Definition implementation – PackageHandler

This is an implementation class where we are defining all the form fields required for the process to run

package com.mysite.mcp.process;

import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.Collections;
import java.util.EnumMap;
import java.util.List;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import com.mysite.mcp.utils;
import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.fs.io.ImportOptions;
import org.apache.jackrabbit.vault.packaging.JcrPackage;
import org.apache.jackrabbit.vault.packaging.JcrPackageManager;
import org.apache.jackrabbit.vault.packaging.PackageException;
import org.apache.jackrabbit.vault.packaging.PackageId;
import org.apache.jackrabbit.vault.packaging.Packaging;
import org.apache.sling.api.resource.LoginException;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.ResourceResolver;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.acs.commons.fam.ActionManager;
import com.adobe.acs.commons.mcp.ProcessDefinition;
import com.adobe.acs.commons.mcp.ProcessInstance;
import com.adobe.acs.commons.mcp.form.FileUploadComponent;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.RadioComponent;
import com.adobe.acs.commons.mcp.model.GenericReport;
import com.adobe.acs.commons.mcp.model.ManagedProcess;

/**
 * Package Creator to create fully functional package for the given path
 */
public class PackageHandler extends ProcessDefinition {

	private static final Logger LOGGER = LoggerFactory.getLogger(PackageHandler.class);
	private final GenericReport report = new GenericReport();
	private static final String REPORT_NAME = "Package_Action_Performed";
	
    public enum PackAction {
        upload, install, upload_install, build, delete
    }

	@FormField(name = "Upload Package", description = "Upload JCR Package", component = FileUploadComponent.class)
	public transient InputStream inputPackage = null;
	
	@FormField(name = "Package Name",
            description = "Package Name to be Installed",
            required = false,
            options = {"default=Package Name"})
    private String packageName;
	
	@FormField(name = "Package Group",
            description = "Package Group to be Installed",
            required = false,
            options = {"default=Package Group"})
    private String packageGroup;
	
	@FormField(name = "Package Version",
            description = "Package Version to be Installed",
            required = false,
            options = {"default=Package Version"})
    private String packageVersion;
	
    @FormField(
            name = "Package Option",
            description = "Option is mandatory to be picked",
            component = RadioComponent.EnumerationSelector.class,
            options = {"default=upload", "vertical"}
    )
    protected transient PackAction packageOption = PackAction.upload;

	ManagedProcess instanceInfo;
	private Packaging packaging;

	public PackageHandler(Packaging packaging) {
		this.packaging = packaging;
	}

	@Override
	public void init() throws RepositoryException {
	}

	@Override
	public void buildProcess(ProcessInstance instance, ResourceResolver resourceResolver)
			throws LoginException, RepositoryException {
		instanceInfo = instance.getInfo();
		instance.getInfo().setDescription("Package "+ packageOption);
		switch (packageOption) {
		case install:
			instance.defineAction("Installing Package", resourceResolver, this::installPackage);
			break;
		case upload_install:
			instance.defineAction("Uploading Package", resourceResolver, this::uploadAndInstallPackage);
			break;
		case upload:
			instance.defineAction("Uploading and Installing Package", resourceResolver, this::uploadPackage);
			break;
		case build:
			instance.defineAction("Building Package", resourceResolver, this::buildPackage);
			break;
		case delete:
			instance.defineAction("Deleting Package", resourceResolver, this::deletePackage);
			break;
		default: 
			break;			
		}
		report.setName(REPORT_NAME);
	}

	protected void uploadPackage(ActionManager manager) {
		manager.deferredWithResolver(this::uploadPack);
	}
	
	protected void installPackage(ActionManager manager) {
		manager.deferredWithResolver(this::installPack);
	}
	
	protected void uploadAndInstallPackage(ActionManager manager) {
		manager.deferredWithResolver(this::uploadAndInstallPack);
	}

	protected void buildPackage(ActionManager manager) {
		manager.deferredWithResolver(this::buildPack);
	}
	
	protected void deletePackage(ActionManager manager) {
		manager.deferredWithResolver(this::deletePack);
	}
	
	private void uploadPack(ResourceResolver resourceResolver) throws RepositoryException, IOException {
		try (InputStream inputPack = inputPackage) {
			if (null != inputPack) {
				final JcrPackageManager packageManager = packaging
						.getPackageManager(resourceResolver.adaptTo(Session.class));
				JcrPackage jcrPackage = packageManager.upload(inputPack, true);
				recordAction(jcrPackage.getNode().getPath(), "Uploaded", "Package Upload is successful");
			}
		}
	}
	
	private void installPack(ResourceResolver resourceResolver) {
		installPackage(resourceResolver, packageGroup, packageName, packageVersion, ImportMode.REPLACE, AccessControlHandling.IGNORE);
	}
	
	private void buildPack(ResourceResolver resourceResolver) {
		buildPackage(resourceResolver, packageGroup, packageName, packageVersion);
	}
	
	private void deletePack(ResourceResolver resourceResolver) {
		deletePackage(resourceResolver, packageGroup, packageName, packageVersion);
	}
	
	private void uploadAndInstallPack(ResourceResolver resourceResolver) throws RepositoryException, IOException {
		try (InputStream inputPack = inputPackage) {
			if (null != inputPack) {
				final JcrPackageManager packageManager = packaging
						.getPackageManager(resourceResolver.adaptTo(Session.class));
				try(JcrPackage jcrPackage = packageManager.upload(inputPack, true)) {
					recordAction(jcrPackage.getNode().getPath(), "Uploaded", "Package Upload is successful");
					installPackage(jcrPackage, ImportMode.REPLACE, AccessControlHandling.IGNORE);
				}				
			}
		}
	}
	
   public boolean installPackage(ResourceResolver resourceResolver, final String groupName,
           final String packageName, final String version, final ImportMode importMode,
           final AccessControlHandling aclHandling) {
       boolean result;
       final JcrPackageManager packageManager = packaging.getPackageManager(resourceResolver.adaptTo(Session.class));
       final PackageId packageId = new PackageId(groupName, packageName, version);
       try(JcrPackage jcrPackage = packageManager.open(packageId)) {
           final ImportOptions opts = VltUtils.getImportOptions(aclHandling, importMode);
           jcrPackage.install(opts);
           result = true;
           recordAction(jcrPackage.getNode().getPath(), "Install", "Package Installation is successful");
       } catch (RepositoryException | PackageException | IOException e) {
           LOGGER.error("Could not install package", e);
           result = false;
       }
       return result;
   }
   
   public boolean buildPackage(ResourceResolver resourceResolver, final String groupName,
           final String packageName, final String version) {
       boolean result;
       final JcrPackageManager packageManager = packaging.getPackageManager(resourceResolver.adaptTo(Session.class));
       final PackageId packageId = new PackageId(groupName, packageName, version);
       try(JcrPackage jcrPackage = packageManager.open(packageId)) {
    	   packageManager.assemble(jcrPackage, null);
           result = true;
           recordAction(jcrPackage.getNode().getPath(), "Build", "Package Building is successful");
       } catch (RepositoryException | PackageException | IOException e) {
           LOGGER.error("Could not install package", e);
           result = false;
       }
       return result;
   }
   
   public boolean deletePackage(ResourceResolver resourceResolver, final String groupName,
           final String packageName, final String version) {
       boolean result;
       final JcrPackageManager packageManager = packaging.getPackageManager(resourceResolver.adaptTo(Session.class));
       final PackageId packageId = new PackageId(groupName, packageName, version);
       try(JcrPackage jcrPackage = packageManager.open(packageId)) {
    	   packageManager.remove(jcrPackage);
           result = true;
           recordAction(jcrPackage.getNode().getPath(), "Delete", "Package Deletion is successful");
       } catch (RepositoryException e) {
           LOGGER.error("Could not install package", e);
           result = false;
       }
       return result;
   }
   
   public boolean installPackage(JcrPackage jcrPackage, final ImportMode importMode, final AccessControlHandling aclHandling) {
       boolean result;
       try {
           final ImportOptions opts = VltUtils.getImportOptions(aclHandling, importMode);
           jcrPackage.install(opts);
           result = true;
       } catch (RepositoryException | PackageException | IOException e) {
           LOGGER.error("Could not install package", e);
           result = false;
       }
       return result;
   }

	public enum ReportColumns {
		PATH, ACTION, DESCRIPTION
	}

	List<EnumMap<ReportColumns, String>> reportData = Collections.synchronizedList(new ArrayList<>());

	private void recordAction(String path, String action, String description) {
		EnumMap<ReportColumns, String> row = new EnumMap<>(ReportColumns.class);
		row.put(ReportColumns.PATH, path);
		row.put(ReportColumns.ACTION, action);
		row.put(ReportColumns.DESCRIPTION, description);
		reportData.add(row);
	}

	@Override
	public void storeReport(ProcessInstance instance, ResourceResolver rr)
			throws RepositoryException, PersistenceException {
		report.setRows(reportData, ReportColumns.class);
		report.persist(rr, instance.getPath() + "/jcr:content/report");
	}
}

Add the Utils class called – VltUtils

package com.mysite.mcp.utils;

import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.fs.io.ImportOptions;

/**
 * Utility class for creating vlt filters and import/export options
 */
public class VltUtils {
	
	public static ImportOptions getImportOptions(AccessControlHandling aclHandling, ImportMode importMode) {
		ImportOptions opts = new ImportOptions();
		if (aclHandling != null) {
			opts.setAccessControlHandling(aclHandling);
		} else {
			// default to overwrite
			opts.setAccessControlHandling(AccessControlHandling.OVERWRITE);
		}
		if (importMode != null) {
			opts.setImportMode(importMode);
		} else {
			// default to update
			opts.setImportMode(ImportMode.UPDATE);
		}

		return opts;
	}
}

Once the code is deployed, please go to the following URL and click on start process as shown below:

http://domain/apps/acs-commons/content/manage-controlled-processes.html

Start MCP Process

You will see a new process called Package Handler as shown below:

Click on the process

Package Handler process

Use the browser to upload the package and use the below options to perform any package-related operations:

  1. Upload
  2. Install
  3. Build
  4. Upload and Install
  5. Delete
Package Handler Option

Use the Package field to browse and upload the package, no need to fill in other options

To Build, Install or delete an existing package please fill in the package name, group, and versions

You can also download the built package by visiting the result page and hitting the path directly over the browser as shown below:

Upload a package and once the packge is installed you will be able to veiw the process result

Process Result page

copy the etc/package result and hot the donain/etc/package to download the package

Package downloaded

Broken Page References AEM


Problem Statement:

How to get the list of all the broken references in AEM?


Requirement:

Get a List of all the broken references using MCP and provide the report


Introduction:

OOTB we get a Broken reference report provided by MCP, which can be used to get all the broken references in the content repo.

Broken Refernce Report

It’s highly recommended to run this process during

  1. off hours
  2. Don’t run on the root level
  3. Run it on 2nd level or 3rd level pages

How to run this process?

Provide Source path

Provide the regex so that it will consider only the references which point to /content or /etc (points to AEM)

You can also provide exclude properties to improve the traversal of nodes.

If you want to verify any broken links in the RTE fields or properties, then check the deep check checkbox and provide the properties list.

But the above process has a few issues.

  1. Html properties are not working as expected

We need a few customizations to this process by making a few changes to check HTML level references by adding JSOUP API

Add the following dependencies to your POM.xml

<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>com.adobe.acs</groupId>
    <artifactId>acs-aem-commons-bundle</artifactId>
    <scope>provided</scope>
</dependency>

Get the following Broken reference code into your local as shown below:

Add the following code as shown below:

if (htmlFields.contains(property.getKey())) {
            stream = stream.flatMap(val -> {
                try {
                    Document doc = Jsoup.parse(val);
                    Elements anchors = doc.select("a");
                    return anchors.stream().map(link -> link.attr("href"));
                } catch (Exception e) {
                    log.warn("Could not parse links from property value of {}", property.getKey(), e);
                    return Stream.empty();
                }
            });
        }
At Line number 207

When we run it on wknd site it would look something like this:

Broken Reference Report

Broken Asset References AEM


Problem Statement:

Get a List of all the Assets which are missing references


Requirement:

Get the list of broken asset references to unpublish and remove them repo to improve the system stability and performance.


Introduction:

How do assets get published?

  1.  The author uploads the images and publishes the assets
  2. Create a launcher and workflow which process assets metadata and publish the pages
  3. Whenever we publish any pages and if the page has references to assets, then during publishing, it asks to replicate the references as well.

What happens when the page is unpublished?

  1.  When the page is deactivated, assets referenced to the page will not be deactivated because this asset might have reference to the other pages hence out of the box assets won’t be deactivated.
  2. If we perform cleanup, deactivate and delete old pages, we might not be cleaning up assets related to this page.

Advantages of cleaning up old assets?

  1. Drastically reduces repository size
  2. Improves DAM Asset search
  3. Improves indexing

Get Publish Report using Assets Report:

  • Go to Tools -> Assets -> Reports as shown below:
Asset Reports
  • Click on create and click on Publish report
Select Publish report
  • Provide folder path and start date and end date
Add Report details
  • Select the columns as per requirement
Configure columns for the report
  • Finally, report will be ready with all the assets lists as shown below
Completed Reports
  • Download the report to see the final list of images
Example Report CSV file

If Images are unpublished then we can ask authors to review and delete them

If images are published but has no references to figure this out, we need a new process.

MCP (Manage Controlled Processes) is both a dashboard for performing complex tasks and a rich API for defining these tasks as process definitions. In addition to kicking off new processes, users can also monitor running tasks, retrieve information about completed tasks, halt work, and so on.

Add the following maven dependency to your pom to extend MCP

<dependency>
    <groupId>com.adobe.acs</groupId>
    <artifactId>acs-aem-commons-bundle</artifactId>
    <version>5.0.4</version>
    <scope>provided</scope>
</dependency>

Broken Asset reference:

Add the following dependencies to your pom.xml

        <dependency>
            <groupId>org.jetbrains</groupId>
            <artifactId>annotations</artifactId>
            <version>17.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.adobe.acs</groupId>
            <artifactId>acs-aem-commons-bundle</artifactId>
            <version>5.2.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.jsoup</groupId>
            <artifactId>jsoup</artifactId>
            <version>1.13.1</version>
            <scope>provided</scope>
        </dependency>

Create a new MCP process by calling the MCP service and providing the implementation class:

package com.mysite.core.mcp;

import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import com.adobe.acs.commons.mcp.ProcessDefinitionFactory;
import com.day.cq.replication.Replicator;

@Component(service = ProcessDefinitionFactory.class)
public class BrokenAssetsFactory extends ProcessDefinitionFactory<BrokenAssets> {

    @Reference
    Replicator replicator;

    @Override
    public String getName() {
        return "Broken Asset References";
    }

    @Override
    public BrokenAssets createProcessDefinitionInstance() {
        return new BrokenAssets(replicator);
    }
}

Create an implementation class as shown below:

package com.mysite.core.mcp;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.EnumMap;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;
import javax.jcr.RepositoryException;
import org.apache.commons.collections4.ListUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.util.Text;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.LoginException;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.jetbrains.annotations.NotNull;
import com.adobe.acs.commons.data.CompositeVariant;
import com.adobe.acs.commons.data.Spreadsheet;
import com.adobe.acs.commons.fam.ActionManager;
import com.adobe.acs.commons.mcp.ProcessDefinition;
import com.adobe.acs.commons.mcp.ProcessInstance;
import com.adobe.acs.commons.mcp.form.Description;
import com.adobe.acs.commons.mcp.form.FileUploadComponent;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.RadioComponent;
import com.adobe.acs.commons.mcp.model.GenericReport;
import com.adobe.acs.commons.mcp.model.ManagedProcess;
import com.day.cq.replication.ReplicationStatus;
import com.day.cq.replication.Replicator;

/**
 * Relocate Pages and/or Sites using a parallelized move process
 */
public class BrokenAssets extends ProcessDefinition {
	
	private static final String SOURCE_PATH = "Path";
	private static final String CONTAINS_QUERY = "CONTAINS(s.*, '%s') or ";
	private final GenericReport report = new GenericReport();
	List<EnumMap<ReportColumns, String>> reportData = Collections.synchronizedList(new ArrayList<>());
	
	List<String> assetsList = new LinkedList<>();
	
    public enum PublishMethod {
        @Description("Select this option to generate Broken References Report")
        BROKEN_REFERENCE_REPORT
    }
    
    Replicator replicatorService;
    
    @FormField(name = "Mapping Excel", component = FileUploadComponent.class)
    private RequestParameter mappingExcel;
    private Spreadsheet spreadsheet;
    
    @FormField(name = "Type",
            description = "Please select the oprporiate option to execute",
            component = RadioComponent.EnumerationSelector.class,
            required = true,
            options = {"vertical", "default=BROKEN_REFERENCE_REPORT"})
    public PublishMethod publishMethod = PublishMethod.BROKEN_REFERENCE_REPORT;

	@FormField(name="Content Path",
			description="Content Path for search results",
			hint="/content",
			options={"default=/content"})
	public String contentPath = "/content";
    
    @FormField(name="Chunk count",
            description="Max number of chunk the search results",
            hint="4500",
            options={"default=3000"})
    public int chunkCount = 3000;
    
    @FormField(name="Retries",
            description="Max number of retries per commit",
            hint="2",
            options={"default=2"})
    public int retryCount = 2;
    
    @FormField(name="Retry delay",
            description="Delay between retries (in milliseconds)",
            hint="500,1000,...",
            options={"default=500"})
    public int retryWait = 500;
    
    public BrokenAssets(Replicator replicator) {
    	replicatorService = replicator;
	}

    @Override
    public void init() throws RepositoryException {    	
    	if(null != mappingExcel && mappingExcel.getSize() > 0) {
    		try {
                // Read spreadsheet
        		spreadsheet = new Spreadsheet(mappingExcel, SOURCE_PATH).buildSpreadsheet();
        		spreadsheet.getDataRowsAsCompositeVariants().forEach(row -> assetsList.add(getString(row, SOURCE_PATH)) );
            } catch (IOException e) {
                throw new RepositoryException("Unable to process spreadsheet", e);
            }
    	}    	    	    	
    }

    ManagedProcess instanceInfo;

    @Override
    public void buildProcess(ProcessInstance instance, ResourceResolver rr) throws LoginException, RepositoryException {
        instanceInfo = instance.getInfo();
        switch (publishMethod.name().toLowerCase()) {
	        case "broken_reference_report":
	        	instance.getInfo().setDescription("Collecting references");
	        	instance.defineAction("Searching Refs", rr, this::collectRefernce);
	            break;
        }        
    }

    @Override
    public void storeReport(ProcessInstance instance, ResourceResolver rr) throws RepositoryException, PersistenceException {    	
        report.setRows(reportData, ReportColumns.class);
        report.persist(rr, instance.getPath() + "/jcr:content/report");
    }

    public enum ReportColumns {
    	ASSET_PATH
    }
            
    protected void collectRefernce(ActionManager manager) {
    	manager.deferredWithResolver(this::collectReferences); 
    }
    
    private void collectReferences(ResourceResolver resourceResolver) throws Exception {    	
		if(!assetsList.isEmpty()) {						 
			RetryUtils.withRetry(retryCount, retryWait, () -> {
			    List<List<String>> chunkedAssetList = ListUtils.partition(assetsList, chunkCount);					
			    int counter = 0;
			    while(!chunkedAssetList.isEmpty() && counter < chunkedAssetList.size()) {
			        List<String> chunk = chunkedAssetList.get(counter);
			        counter++;
			        @NotNull Iterator<Resource> resourceResults = buildSQLQueryAndFetchResults(resourceResolver, contentPath, chunk);
			        collectReferences(resourceResults, chunk);
			    }
			});	
			if(!assetsList.isEmpty()) {
				assetsList.stream().forEach(this::reportResult);
			}
		}	        			
    }

	private void collectReferences(@NotNull Iterator<Resource> resourceResults, List<String> chunk) {
		while(resourceResults.hasNext()) {
			Resource resultRes = resourceResults.next();
			if (null != resultRes) {
				resultRes.getValueMap().entrySet().forEach(property -> {
					if(!property.getKey().equalsIgnoreCase("dam:folderThumbnailPaths")) {
						Object prop = property.getValue();
						if (prop.getClass() == String[].class) {
							List<String> propertyValue = Arrays.asList((String[]) prop);
							if (!propertyValue.isEmpty()) {
								List<String> matchingAsset = chunk.stream().filter(
										assetPat -> propertyValue.stream().anyMatch(sam -> sam.contains(assetPat)))
										.collect(Collectors.toList());
								if(!matchingAsset.isEmpty()) {
									chunk.removeIf(matchingAsset::contains);
									assetsList.removeIf(matchingAsset::contains);
								}
							}
						} else if (prop.getClass() == String.class) {
							String propertyValue = (String) prop;
							if (StringUtils.isNotEmpty(propertyValue) ) {
								Optional<String> matchingAsset = chunk.stream().filter(propertyValue::contains).findAny();
								if(matchingAsset.isPresent()) {
									chunk.remove(matchingAsset.get());
									assetsList.remove(matchingAsset.get());
								}
							}
						}
					}
				});
			}
		}
	}

	private void reportResult(String path) {
		EnumMap<ReportColumns, String> row = new EnumMap<>(ReportColumns.class);
		row.put(ReportColumns.ASSET_PATH, path);		
		reportData.add(row);
	}	
	
	private @NotNull Iterator<Resource> buildSQLQueryAndFetchResults(ResourceResolver resolver, String contentPath, List<String> chunk) {
        String groupStr = chunk.stream().map(r -> String.format(CONTAINS_QUERY, Text.escapeIllegalXpathSearchChars(r).replaceAll("'", "''"))).collect(Collectors.joining());        
		String querySt = "SELECT * FROM [nt:base] AS s WHERE ISDESCENDANTNODE(["+contentPath+"]) and "+ StringUtils.substringBeforeLast(groupStr, "or");
		return resolver.findResources(querySt, "JCR-SQL2");
    }
	
	@SuppressWarnings({"rawtypes", "unchecked"})
    private String getString(Map<String, CompositeVariant> row, String path) {
        CompositeVariant v = row.get(path.toLowerCase(Locale.ENGLISH));
        if (v != null) {
            return (String) v.getValueAs(String.class);
        } else {
            return null;
        }
    }
}
package com.mysite.core.mcp;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryUtils {
    private static final Logger log = LoggerFactory.getLogger(RetryUtils.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int i = 0; i < maxTimes; i++) {
            try {
                call.process();
                return true;
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (i + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

Once the code is deployed, please go to the following URL and click on start process as shown below:

http://{domain}/apps/acs-commons/content/manage-controlled-processes.html

Running Asset Reference Process:

After building the code you can see the new Process showing up in MCP

Borken Asset Refernce Process

Copy the Path column into the new Excel sheet as shown below

Path column into new excel file

Upload into the process and start to see all the images which are published yet unreferenced as shown below

Why does Chunk count?

Chunk count helps the SQL 2 query to group by the paths, which will be maxing 4500 and it won’t take more than that (configurable based on the environment). However basically, if we have 20000 / 4500 = 4.44 ~ 5 we will be running the query max five times to generate the below report

Share the report with the content authors team to validate if images are required if not plan to clean up using

After completing the process
Example report and we can download and share with Authors

Clean up Process:

Authors don’t have to unpublish and delete individual images, then can you use the below process to upload the excel sheet with all the approved image paths and upload it to the process to deactivate and delete

AEM Publish / UnPublish / Delete List of pages – MCP Process

Unclosed resource resolver complete guide AEM


Problem Statement:

Unclosed resource resolver issue is causing performance impact on the AEM environment

It’s hard to debug and fix the resolver issues

Best practices to close the resolver


Requirement:

Provide all the best practices to close the unclosed resource resolve and in turn improve environment stability.


Introduction:

Usually, we create/open a resource resolver:

  1. Servlets
  2. OSGi Services
  3. Workflows
  4. Schedulers

Purpose of service user based resolver.

To get access to the paths which is blocked for every one group, make changes and commit changes etc.

Creating Resource resolver instance:

This feature was released with Java 7 and the try-with-resources statement is a try statement that declares one or more resources. A resource is an object that must be closed after the program is finished with it. The try-with-resources statement ensures that each resource is closed at the end of the statement. Any object that implements java.lang.AutoCloseable, which includes all objects which implement java.io.Closeable, can be used as a resource.

So basically, creating the resolver within the calling class like below will auto close:

try (ResourceResolver resourceResolver = resolverFactory.getServiceResourceResolver(Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "{service user name}"))){
    //business logic
}

And it is highly recommended to use the following APIs as well:

  1. FileReader
  2. ZipFile
  3. BufferedWriter

etc.

Closing the resolver for Queries:

Whenever we come across queries, we need to close the resolver manually even after using the try resource approach:

Try resource Approach:

ResourceResolver resolverResolverLeakingReference = null;
try (ResourceResolver resolver = resourceResolverFactory.getServiceResourceResolver(Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "{service user name}"))) {
    session = resolver.adaptTo(Session.class);
    queryBuilder = resolver.adaptTo(QueryBuilder.class);
    
    PredicateGroup predicateGroup = null;
    try {
        predicateGroup = PredicateGroup.create(map);
    } catch (VerifyError ex) {
        predicateGroup = new PredicateGroup();
    }

    Query query = queryBuilder.createQuery(predicateGroup, session);
    SearchResult result = query.getResult();
    for (final Hit hit : result.getHits()) {
        Resource resource = hit.getResource();            	
        if(resolverResolverLeakingReference == null){
            resolverResolverLeakingReference =  resource.getResourceResolver();
        }    
        //buisness logic                            
    }
} catch (Exception e) {
    log.error("Error fetching locations count results", e);
} finally {
    if (resolverResolverLeakingReference != null) {
        // Always Close the leaking QueryBuilder resourceResolver.
        resolverResolverLeakingReference.close();    
    }        
}

Requet.getResolver Approach:

ResourceResolver leakingResourceResolverReference = null;
try {
    Session session = request.getResourceResolver().adaptTo(Session.class);
    Map<String, String> map = new HashMap<>();
    //map put goes here
        Query query = queryBuilder.createQuery(PredicateGroup.create(map), session);
        SearchResult result = query.getResult();
        for (Hit hit : result.getHits()) {
            if(leakingResourceResolverReference == null) {
                leakingResourceResolverReference =  hit.getResource().getResourceResolver();
            }		         
            //buisness logic
        }    
} catch(Exception e) {
    log.info("error::{}",e.getMessage());
} finally {
    if(leakingResourceResolverReference != null){
        leakingResourceResolverReference.close();
    }
}

Or you can also follow this:

SearchResult result;
try {
    Session session = request.getResourceResolver().adaptTo(Session.class);
    Map<String, String> map = new HashMap<>();
    //map put goes here
        Query query = queryBuilder.createQuery(PredicateGroup.create(map), session);
        result = query.getResult();
        for (Hit hit : result.getHits()) {    
            //buisness logic
        }    
} catch(Exception e) {
    log.info("error::{}",e.getMessage());
} finally {
    Iterator<Resource> resources = result.getResources();
    if (resources.hasNext()) {
        resources.next().getResourceResolver().close();
    }
}

Avoid query builder and use SQL2:

try (ResourceResolver resourceResolver = resolverFactory.getServiceResourceResolver(Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "{service user name}"))){
    String querySt = "SELECT * FROM [nt:base] ";		
    Iterator<Resource> results = resourceResolver.findResources(querySt, "JCR-SQL2");
    //buisness logc
}

Use Java Streams:

You can follow the below blog to use Java stream for executing queries in AEM

AEM Query builder using Java streams

Delete Old CRX Packages AEM


Problem statement:

AEM environment size is increasing because of user-generated packages


Requirement:

Can we purge all the user-generated packages to improve stability?


Introduction:

A package is a zip file holding repository content in the form of a file-system serialization (called “vault” serialization). This provides an easy-to-use-and-edit representation of files and folders.

Packages include content, both page content and project-related content, selected using filters.

A package also contains vault meta information, including the filter definitions and import configuration information. Additional content properties (that are not used for package extraction) can be included in the package, such as a description, a visual image, or an icon; these properties are for the content package consumer and for informational purposes only.

Usually, Developers or AEM content synch or even code deployment will keep on piling up in the CRX packages and it will be consuming spaces on MBs and even sometimes GBs.

If we move more packages, then loading crx/packmgr would more time.

Hence you can create a scheduler that runs on off-hours which cleans up the packages and which will get back the space and avoids extra maintenance tasks.

The below scheduler will remove all the packages for my_package group we can add business logic to handle for other groups

package com.mysite.core.schedulers;
import org.osgi.service.metatype.annotations.AttributeDefinition;
import org.osgi.service.metatype.annotations.AttributeType;
import org.osgi.service.metatype.annotations.ObjectClassDefinition;
@ObjectClassDefinition(name = "Old Packages Purge Schedular", description = "Remove old packages from different paths")
public @interface PurgeOldPackagesSchedulerConfig {
    String DEFAULT_SCHEDULER_EXPRESSION = "0 0 16 ? * SUN *"; // every Sunday 4 PM
    boolean DEFAULT_SCHEDULER_CONCURRENT = false;
    @AttributeDefinition(name = "Enabled", description = "True, if scheduler service is enabled", type = AttributeType.BOOLEAN)
    boolean enabled() default true;
    @AttributeDefinition(name = "Cron expression defining when this Scheduled Service will run", description = "[every minute = 0 * * * * ?], [12:00am daily = 0 0 0 ? * *]", type = AttributeType.STRING)
    String schedulerExpression() default DEFAULT_SCHEDULER_EXPRESSION;
    @AttributeDefinition(name = "package paths", description = "package folder paths", type = AttributeType.STRING)
    String[] packagesPaths() default {"my_packages"};
}
package com.mysite.core.schedulers;
import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import org.apache.jackrabbit.vault.packaging.JcrPackage;
import org.apache.jackrabbit.vault.packaging.JcrPackageManager;
import org.apache.jackrabbit.vault.packaging.Packaging;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceResolverFactory;
import org.apache.sling.commons.scheduler.ScheduleOptions;
import org.apache.sling.commons.scheduler.Scheduler;
import org.osgi.service.component.annotations.Activate;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.ConfigurationPolicy;
import org.osgi.service.component.annotations.Deactivate;
import org.osgi.service.component.annotations.Modified;
import org.osgi.service.component.annotations.Reference;
import org.osgi.service.metatype.annotations.Designate;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Component(immediate = true, service = PurgeOldPackagesScheduler.class, configurationPolicy = ConfigurationPolicy.REQUIRE)
@Designate(ocd = PurgeOldPackagesSchedulerConfig.class)
public class PurgeOldPackagesScheduler implements Runnable {
    private static final Logger log = LoggerFactory.getLogger(PurgeOldPackagesScheduler.class);
    /**
     * Id of the scheduler based on its name
     */
    private String schedulerJobName;
    Session session;
    private List<String> packagesPathsList;
    SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
    @Reference
    private Packaging packaging;
    @Reference
    private Scheduler scheduler;
    @Reference
    private ResourceResolverFactory resolverFactory;
    @Activate
    @Modified
    protected void activate(PurgeOldPackagesSchedulerConfig purgeOldPackagesSchedulerConfig) {
        /**
         * Creating the scheduler id
         */
        this.schedulerJobName = this.getClass().getSimpleName();
        addScheduler(purgeOldPackagesSchedulerConfig);
        packagesPathsList = Arrays.asList(purgeOldPackagesSchedulerConfig.packagesPaths());
    }
    /**
     * @see Runnable#run().
     */
    @Override
    public final void run() {
        log.debug("PurgeOldPackagesScheduler Job started");
        purgeOldPackages(packagesPathsList);
        log.debug("PurgeOldPackagesScheduler Job completed");
    }
    void purgeOldPackages(List<String> packagesPathsList) {
        try (ResourceResolver resourceResolver = resolverFactory.getServiceResourceResolver(Collections
                .singletonMap(ResourceResolverFactory.SUBSERVICE, ServiceUserConstants.ADMINISTRATIVE_SERVICE_USER))) {
            session = resourceResolver.adaptTo(Session.class);
            JcrPackageManager jcrPackageManager = packaging.getPackageManager(session);
            packagesPathsList.forEach(group -> {
                removePackages(jcrPackageManager, 0, group);
            });
        } catch (Exception e) {
            log.error(e.getMessage(), e);
        } finally {
            if (session != null) {
                session.logout();
            }
        }
    }
    @Deactivate
    protected void deactivate() {
        /**
         * Removing the scheduler
         */
        removeScheduler();
    }
    /**
     * This method adds the scheduler
     *
     * @param purgeOldPackagesSchedulerConfig
     */
    private void addScheduler(PurgeOldPackagesSchedulerConfig purgeOldPackagesSchedulerConfig) {
        /**
         * Check if the scheduler is enabled
         */
        if (purgeOldPackagesSchedulerConfig.enabled()) {
            /**
             * Scheduler option takes the cron expression as a parameter and run accordingly
             */
            ScheduleOptions scheduleOptions = scheduler.EXPR(purgeOldPackagesSchedulerConfig.schedulerExpression());
            /**
             * Adding some parameters
             */
            scheduleOptions.name(schedulerJobName);
            scheduleOptions.canRunConcurrently(false);
            /**
             * Scheduling the job
             */
            scheduler.schedule(this, scheduleOptions);
            log.info("{} Scheduler added", schedulerJobName);
        } else {
            log.info("Scheduler is disabled");
            removeScheduler();
        }
    }
    /**
     * This method removes the scheduler
     */
    private void removeScheduler() {
        log.info("Removing scheduler: {}", schedulerJobName);
        /**
         * Unscheduling/removing the scheduler
         */
        scheduler.unschedule(String.valueOf(schedulerJobName));
    }
    private void removePackages(JcrPackageManager jcrPackageManager, int counterStart, String groupName) {
        try {
            List<JcrPackage> packages = jcrPackageManager.listPackages(groupName, false);
            AtomicInteger counter = new AtomicInteger(counterStart);
            if (null != packages && !packages.isEmpty()) {
                packages.stream().sorted(Comparator.nullsLast((e1, e2) -> {
                    try {
                        return ((JcrPackage) e1).getPackage().getCreated()
                                .compareTo(((JcrPackage) e2).getPackage().getCreated());
                    } catch (RepositoryException | IOException ex) {
                        log.error(ex.getMessage());
                    }
                    return -1;
                }).reversed()).forEach(pack -> {
                    if (counter.incrementAndGet() > 3) {
                        try {
                            jcrPackageManager.remove(pack);
                        } catch (RepositoryException ex) {
                            log.error(ex.getMessage());
                        }
                    }
                });
            }
        } catch (RepositoryException e) {
            e.printStackTrace();
        }
    }
}

Caching AEM GraphQL queries with content fragment


Problem Statement:

How can I persist query?

How to cache my query results?
How to Update my queries?


Requirement:

Provide details on how to add the persist graphql query, cache the results from graphql and update the persisted query

Provide curl commands to execute in terminal or on postman


Introduction:

Persisted Queries (Caching)

After preparing a query with a POST request, it can be executed with a GET request that can be cached by HTTP caches or a CDN.

This is required as POST queries are usually not cached, and if using GET with the query as a parameter there is a significant risk of the parameter becoming too large for HTTP services and intermediates.

Persisted queries must always use the endpoint related to the appropriate Sites configuration; so, they can use either, or both:

  • Specific Sites configuration and endpoint

Creating a persisted query for a specific Sites configuration requires a corresponding Sites-configuration-specific endpoint (to provide access to the related Content Fragment Models).

For example, to create a persisted query specifically for the SampleGraphQL Sites configuration:

a corresponding SampleGraphQL-specific Sites configuration

  • Go to the tools section for the aem and general section and select Configuration Browser as shown below
Configuration browser
  • Add select the conf folder and go to the properties and make GraphQL Persistent Queries checkbox is checked
Enable persistent queries

a SampleGraphQL-specific endpoint must be created in advance.

  • Go to tools section for the aem and assets section and select GraphQL as shown below
assets -> graphql
  • Add the new end point as shown below:
endpoint

Add the following CORS configurations for the GraphQL API calls:

CORS config

Register graphql search path:

Register Servlet path

Here are the steps required to persist a given query:

Prepare the query by putting it to the new endpoint URL /graphql/persist.json/<config>/<persisted-label>.

For example, create a persisted query:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/cities' \
--header 'Content-Type: application/json' \
--data-raw '{
  cityList {
    items {
      _path
      name
      country
      population
    }
  }
}'
  • At this point, check the response.

For example, check for success:

{
    "action": "create",
    "configurationName": "SampleGraphQL",
    "name": "cities",
    "shortPath": "/SampleGraphQL/cities",
    "path": "/conf/SampleGraphQL/settings/graphql/persistentQueries/cities"
}

You can then replay the persisted query by getting the URL /graphql/execute.json/<shortPath>.

For example, use the persisted query:

curl -u admin:admin -X GET 'http://localhost:4502/graphql/execute.json/SampleGraphQL/cities' \
--header 'Authorization: Basic YWRtaW46YWRtaW4='

Update a persisted query by POSTing to an already existing query path.

For example, use the persisted query:

curl -u admin:admin -X POST 'http://localhost:4502/graphql/persist.json/SampleGraphQL/cities' \
--header 'Content-Type: application/json' \
--data-raw '{
  cityList {
    items {
      _path
      name
      country
      population
    }
  }
}'

Create a wrapped plain query.

For example:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/plain-cities' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "{cityList { items { _path name country country population } } }"}'

Create a wrapped plain query with cache control.

For example:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/plain-cities-max-age' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "{cityList { items { _path name country country population } } }", "cache-control": { "max-age": 300 }}'

Create a persisted query with parameters:

For example:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/plain-cities-query-parameters' \
--header 'Content-Type: application/json' \
--data-raw \
'query GetAsGraphqlModelTestByPath($apath: String!) {
    cityByPath(_path: $apath) {
        item {
        _path
        name
        country
        population
        }
    }
  }'

Executing a query with parameters.

For example:

curl -u admin:admin -X POST \
    -H "Content-Type: application/json" \
    "http://localhost:4502/graphql/execute.json/SampleGraphQL/plain-cities-query-parameters;apath=%2Fcontent%2Fdam%2Fsample-content-fragments%2Fcities%2Fberlin"

curl -u admin:admin -X GET \
    "http://localhost:4502/graphql/execute.json/SampleGraphQL/plain-cities-query-parameters;apath=%2Fcontent%2Fdam%2Fsample-content-fragments%2Fcities%2Fberlin"

Bulk Add, Update and Delete properties in AEM – without using Groovy console

Problem Statement:

How to Bulk Add, Update or remove page properties in AEM? Without using the Groovy console.

Requirement:

Create a reusable process that can be used to search for the pages based on resourceType and do the CRUD operations on the results.

Introduction:

Usually, whenever we are using editable templates, we might have some initial content but for some reason, if we want to update the experience fragment path or some page properties then usually, we go for Groovy script to run bulk update.

But AMS don’t install developer tools on the PROD, we need to go to other options and for the above requirement, we can use MCP.

MCP (Manage Controlled Processes) is both a dashboard for performing complex tasks and a rich API for defining these tasks as process definitions. In addition to kicking off new processes, users can also monitor running tasks, retrieve information about completed tasks, halt work, and so on.

Add the following maven dependency to your pom to extend MCP

<dependency>
    <groupId>com.adobe.acs</groupId>
    <artifactId>acs-aem-commons-bundle</artifactId>
    <version>5.0.4</version>
    <scope>provided</scope>
</dependency>

Create Process Definition factory – PropertyUpdateFactory

This class tells ACS Commons MCP to pick the process definition and process name getName and you need to mention the implementation class inside the createProcessDefinitionInstance method as shown below:

package com.mysite.mcp.process;

import org.osgi.service.component.annotations.Component;
import com.adobe.acs.commons.mcp.ProcessDefinitionFactory;

@Component(service = ProcessDefinitionFactory.class, immediate = true)
public class PropertyUpdateFactory extends ProcessDefinitionFactory<PropertyUpdater> {
    @Override
    public String getName() {
        return "Property Updator";
    }
    @Override
    protected PropertyUpdater createProcessDefinitionInstance() {
        return new PropertyUpdater();
    }
}

Create Process Definition implementation – PropertyUpdater

This is an implementation class where we are defining all the form fields required for the process to run

package com.mysite.mcp.process;

import java.io.IOException;
import java.text.MessageFormat;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Collections;
import java.util.EnumMap;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import javax.jcr.RepositoryException;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.LoginException;
import org.apache.sling.api.resource.ModifiableValueMap;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceUtil;
import org.apache.sling.resource.filter.ResourceFilterStream;
import org.jetbrains.annotations.NotNull;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.acs.commons.data.Spreadsheet;
import com.adobe.acs.commons.fam.ActionManager;
import com.adobe.acs.commons.mcp.ProcessDefinition;
import com.adobe.acs.commons.mcp.ProcessInstance;
import com.adobe.acs.commons.mcp.form.FileUploadComponent;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.SelectComponent;
import com.adobe.acs.commons.mcp.form.TextfieldComponent;
import com.adobe.acs.commons.mcp.model.GenericReport;
import com.day.cq.commons.jcr.JcrConstants;

public class PropertyUpdater extends ProcessDefinition {

    private static final Logger LOGGER = LoggerFactory.getLogger(PropertyUpdater.class);
    private final GenericReport report = new GenericReport();
    private static final String REPORT_NAME = "Property-update-report";
    private static final String RUNNING = "Running ";
    private static final String EXECUTING_KEYWORD = " Property Updation";

    private static final String PROPERTY = "property";
    private static final String PROPERTY_TYPE = "type";
    private static final String PROPERTY_VALUE = "value";

    protected enum UpdateAction {
        ADD, UPDATE, DELETE
    }

    @FormField(name = "Property Update Excel", component = FileUploadComponent.class)
    private RequestParameter sourceFile;

    @FormField(name = "Path",
            component = TextfieldComponent.class,
            hint = "(Provide the Relative path to start search)",
            description = "A query will be executed starting this path")
    private String path;

    @FormField(name = "Action",
            component = SelectComponent.EnumerationSelector.class,
            description = "Add, Update or Delete?",
            options = "default=Add")
    UpdateAction reAction = UpdateAction.ADD;

    @FormField(name = "ResourceType",
            component = TextfieldComponent.class,
            hint = "(Provide the resourcetype to be searched for)",
            description = "A query will be executed based on resourcetype")
    private String pageResourceType;

    private Map<String, Object> propertyMap = new HashMap<>();

    @Override
    public void init() throws RepositoryException {
        validateInputs();
    }

    @Override
    public void buildProcess(ProcessInstance instance, ResourceResolver rr) throws LoginException, RepositoryException {
        report.setName(REPORT_NAME);
        instance.getInfo().setDescription(RUNNING + reAction + EXECUTING_KEYWORD);
        instance.defineCriticalAction("Adding the Properties", rr, this::updateProperties);
    }

    private void updateProperties(ActionManager manager) {
        manager.deferredWithResolver(this::addProperties);
    }

    private void addProperties(ResourceResolver resourceResolver) {
        @NotNull Resource resource = resourceResolver.resolve(path);
        if(!ResourceUtil.isNonExistingResource(resource)) {
            ResourceFilterStream rfs = resource.adaptTo(ResourceFilterStream.class);
            rfs.setBranchSelector("[jcr:primaryType] == 'cq:Page'")
                    .setChildSelector("[jcr:content/sling:resourceType] == $type")
                    .addParam("type", pageResourceType)
                    .stream()
                    .map(r -> r.getChild(JcrConstants.JCR_CONTENT))
                    .forEach(this::updateProperty);
        }
    }

    private void updateProperty(Resource resultResource) {
        if (reAction == UpdateAction.ADD || reAction == UpdateAction.UPDATE) {
            addOrUpdateProp(resultResource);
        } else if (reAction == UpdateAction.DELETE) {
            removeProp(resultResource);
        }
    }

    private void removeProp(Resource resultResource) {
        try {
            ModifiableValueMap map = resultResource.adaptTo(ModifiableValueMap.class);
            propertyMap.entrySet().stream().forEach(r -> map.remove(r.getKey()));
            resultResource.getResourceResolver().commit();
            recordAction(resultResource.getPath(), reAction.name(), StringUtils.join(propertyMap));
        } catch (PersistenceException e) {
            LOGGER.error("Error occurred while persisting the property {}", e.getMessage());
        }
    }

    private void addOrUpdateProp(Resource resultResource) {
        try {
            ModifiableValueMap map = resultResource.adaptTo(ModifiableValueMap.class);
            propertyMap.entrySet().stream().forEach(r -> map.put(r.getKey(), r.getValue()));
            resultResource.getResourceResolver().commit();
            recordAction(resultResource.getPath(), reAction.name(), StringUtils.join(propertyMap));
        } catch (PersistenceException e) {
            LOGGER.error("Error occurred while persisting the property {}", e.getMessage());
        }
    }

    public enum ReportColumns {
        PATH, ACTION, DESCRIPTION
    }

    private void validateInputs() throws RepositoryException {
        if (sourceFile != null && sourceFile.getSize() > 0) {
            Spreadsheet sheet;
            try {
                sheet = new Spreadsheet(sourceFile, PROPERTY, PROPERTY_TYPE, PROPERTY_VALUE).buildSpreadsheet();
            } catch (IOException ex) {
                throw new RepositoryException("Unable to parse spreadsheet", ex);
            }

            if (!sheet.getHeaderRow().contains(PROPERTY) || !sheet.getHeaderRow().contains(PROPERTY_TYPE) || !sheet.getHeaderRow().contains(PROPERTY_VALUE)) {
                throw new RepositoryException(MessageFormat.format("Spreadsheet should have two columns, respectively named {0}, {1} and {2}", PROPERTY, PROPERTY_TYPE, PROPERTY_VALUE));
            }

            sheet.getDataRowsAsCompositeVariants().forEach(row -> {
                String propertyType = row.get(PROPERTY_TYPE).toString();
                if(StringUtils.equalsAnyIgnoreCase("String", propertyType)) {
                    propertyMap.put(row.get(PROPERTY).toString(), row.get(PROPERTY_VALUE).toString());
                } else if(StringUtils.equalsAnyIgnoreCase("Date", propertyType)) {
                    SimpleDateFormat dt = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS");
                    propertyMap.put(row.get(PROPERTY).toString(), dt.format(row.get(PROPERTY_VALUE).toString()));
                } else if(StringUtils.equalsAnyIgnoreCase("Array", propertyType)) {
                    propertyMap.put(row.get(PROPERTY).toString(), row.get(PROPERTY_VALUE).toString().split(","));
                } else if(StringUtils.equalsAnyIgnoreCase("Long", propertyType)) {
                    Integer result = Optional.ofNullable(row.get(PROPERTY_VALUE).toString())
                            .filter(Objects::nonNull)
                            .map(Integer::parseInt)
                            .orElse(0);
                    propertyMap.put(row.get(PROPERTY).toString(), result);
                } else if(StringUtils.equalsAnyIgnoreCase("Binary", propertyType)) {
                    propertyMap.put(row.get(PROPERTY).toString(), Boolean.valueOf(row.get(PROPERTY_VALUE).toString()));
                }
            });
        }
    }

    List<EnumMap<ReportColumns, String>> reportData = Collections.synchronizedList(new ArrayList<>());

    private void recordAction(String path, String action, String description) {
        EnumMap<ReportColumns, String> row = new EnumMap<>(ReportColumns.class);
        row.put(ReportColumns.PATH, path);
        row.put(ReportColumns.ACTION, action);
        row.put(ReportColumns.DESCRIPTION, description);
        reportData.add(row);
    }

    @Override
    public void storeReport(ProcessInstance instance, ResourceResolver rr) throws RepositoryException, PersistenceException {
        report.setRows(reportData, ReportColumns.class);
        report.persist(rr, instance.getPath() + "/jcr:content/report");
    }
}

Once code is deployed, please go to the following URL and click on start process as shown below:

http://{domain}/apps/acs-commons/content/manage-controlled-processes.html

Start MCP Process

You will see a new process called Property Update as shown below:

Click on the process

Property Updater process

Upload the excel sheet and provide search Path, Action and Resource Type to search for

Configure Process

Excel Sheet:

Upload the excel sheet attached and add the property name, data type and value as shown below:

Property Updater Excel

After executing the process, you can click the completed process and view the results as shown below:

View report
List of updated results with Action

You can also go to CRXDE and check the property getting added to the resources as shown below:

CRXDE results

You can also perform other options by selecting appropriate Action as shown below:

Action Dropdown

You can learn more about MCP here

AEM Content Fragments with GraphQL – Getting started with GraphQL

Problem Statement:

What is GraphQL?

How GraphQL can be used with Content Fragments?

Introduction:

What is GraphQL?

GraphQL is a query language for APIs and provides a complete and understandable description of the data in your API.

For example:

Let’s consider an external system with the following tables:

1 to 1 relationship between Company table with person table and 1 to 1 person table with awards table.

If I need to get all awards

You might be doing a call to

{domain}/api/awards

awards API flow

To get individual person and awards

{domain}/api/persons?personID{ID}&awards={ID}

person API flow

To get individual company and person and awards

{domain}/api/company?companyNam={NAME}personID={ID}&awards={ID}

Company API flow

But in GraphQL you can send the parameters like a query and get all the related content as well

GraphQL API flow

To use Graph QL you need to prepare schemas and based on the schema you can do filter the data.

For more information on GraphQL, you can be visiting the link

Benefits:

  • Avoiding iterative API requests as with REST,
  • Ensuring that delivery is limited to the specific requirements,
  • Allowing for bulk delivery of exactly what is needed for rendering as the response to a single API query.

How GraphQL can be used with Content Fragments?

GraphQL is a strongly typed API, which means that data must be clearly structured and organized by type.

The GraphQL specification provides a series of guidelines on how to create a robust API for interrogating data on a certain instance. To do this, a client needs to fetch the Schema, which contains all the types necessary for a query.

For Content Fragments, the GraphQL schemas (structure and types) are based on Enabled Content Fragment Models and their data types.

Content Fragments can be used as a basis for GraphQL for AEM queries as:

  • They enable you to design, create, curate and publish page-independent content.
  • The Content Fragment Models provide the required structure by means of defined data types.
  • The Fragment Reference, available when defining a model, can be used to define additional layers of structure.
Model References provided by Adobe

Content Fragments

  • Contain structured content.
  • They are based on a Content Fragment Model, which predefines the structure for the resulting fragment.

Content Fragment Models

  • Are used to generate the Schemas, once Enabled.
  • Provide the data types and fields required for GraphQL. They ensure that your application only requests what is possible, and receives what is expected.
  • The data type Fragment References can be used in your model to reference another Content Fragment, and so introduce additional levels of structure.

Fragment References

  • Is of particular interest in conjunction with GraphQL.
  • Is a specific data type that can be used when defining a Content Fragment Model.
  • References another fragment, dependent on a specific Content Fragment Model.
  • Allows you to retrieve structured data.
    • When defined as a multifeed, multiple sub-fragments can be referenced (retrieved) by the prime fragment.

JSON Preview

To help with designing and developing your Content Fragment Models, you can preview JSON output.

Install:

  1. AEM 6.5.11 (aem-service-pkg-6.5.11.zip)
  2. Graph QL OAK Index (cfm-graphql-index-def-1.0.0.zip)
  3. GraphiQL Developer tool (graphiql-0.0.6.zip)

For AEMacS you will get the content fragment with the latest update.

Go to configuration folder

  1. AEM tools section
  2. General selection in sidebar
  3. Configuration bowser

As shown below:

Configuration Folder

Create a configuration folder and select

  1. Content Fragment Models
  2. GraphQL Persistent Queries

As shown below:

Create a Conf folder with required checkboxes

Go to Assets Model:

  1. AEM tools section
  2. Assets selection in sidebar
  3. Content Fragments Model

As shown below:

Go to Assets CF

Select the folder and create the content fragments as shown below:

CF models

You can also install the package attached here

Go to the following URL to access the GraphiQL developer tool and run the following query:

Note: you can also get all the autosuggestions by using the ctrl+space shortcut

{
  cityByPath(_path: "/content/dam/sample-content-fragments/cities/berlin") {
    item {
      _path
      name
      country
      population
      categories
    }
  }
}
GraphiQL developer tool

Download Sample Package here

You can also find more queries and filters in the following link