Delete Unreferenced Assets AEM

Problem Statement:

Delete all the assets which don’t have references to improve AEM performance in turn Indexes and search/query performance.

Introduction:

How do assets get published?

  1. The author uploads the images and publishes the assets
  2. Create a launcher and workflow which processes assets metadata and publishes the pages
  3. Whenever we publish any pages and if the page has references to assets, then during publishing, it asks to replicate the references as well.

What happens when the page is unpublished?

  1. When the page is deactivated, assets referenced to the page will not be deactivated because this asset might have reference to the other pages hence out of the box assets won’t be deactivated.
  2. If we perform cleanup, deactivate and delete old pages, we might not be cleaning up assets related to this page.

What advantages of cleaning up of old assets?

  1. Drastically reduces repository size
  2. Improves DAM Asset search
  3. Improves indexing

Generate Published Asset Report by visiting:

Go to Tools -> Assets -> Reports as shown below:

Asset Report Tool Section

Click on create and click on Publish report

Asset Publish Report

Provide folder path and start date and end date

Asset Report Configure Page

Select the columns as per the requirement

Asset Report Custom Column Page

Finally, the report will be ready with all the assets lists as shown below

Asset Report

Download the report to see the final list of images

DAM Report result

To generate the report let’s create a new AEM tool:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:
  1. AEM operations
  2. Managing templates
  3. Cloud configurations
  4. ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Broken Asset Report generates a report of all the unreferenced assets by running a Reference Search query across the repository every 30s (by default), you can update the scheduler expression based on your repository size.

The scheduler also checks for the CPU/HEAP size before triggering the reference search process and for more details on Throttled scheduler please refer to the link.

In order to create a perfect content backup package please use the Tool generator to give your tool name and descriptions: https://kiransg.com/2022/11/24/aem-tool-create-generate-tool-from-scratch

Please refer to my GitHub repository for working code on Broken Asset Reference:

Once the repository is built and deployed, you will be able to access the Broken Asset Reference report as shown below:

Broken Asset Reference Tool Section

You can select the report just now created by you in the drop down as shown below:

Select the Asset Report from dropdown

Provide scheduler expression as per your needs and select the result refresh interval for every 10s or as per your needs

You can see the results as shown below once the process is kicked off and it will also show the current row its processing and also CPU/Heap usage.

Reference Search Running status

For some reason, if your system CPU/Heap is throttling then from the backend it takes care of not running your scheduler or you can also manually unschedule the scheduler.

Once your system’s CPU comes back to normal you can go back and select the report and schedule again and report generation picks from the current row where it was left off.

Once the processing is complete click on the report name to download the generated report.

Reference Search Completed Status
Report Excel with Has reference Column

Cross verification:

  1. You can rerun the generated report on the MCP broken asset reference.
  2. Generate the Splunk (logs) results by running a query to get all the assets to call (/content/dam) requests on dispatcher/publisher from the past 1year or so.
  3. You can also reach out to the Analytics team, requesting image impressions (data on image usage) from the past 1 year or so.

Please provide your valuable feedback in the comments.

AEM Performance Optimization CPU/HEAP Status

Problem Statement:

As a developer or user, I would like to make an informed decision by fetching the AEM system CPU/Heap status before or while running a process.

Introduction:

Use cases for developers:

  1. Infinite loops – a coding error
  2. Garbage collection is not handled – unclosed streams
  3. An exception like out of bound issue
  4. Heap size issue – saving loads of data or declaring/manipulating too many strings

Java MX Bean is an API that provides detailed information on JVM CPU/MEM (Heap) status.

A platform MXBean is a managed bean that conforms to the JMX Instrumentation Specification and only uses a set of basic data types. A JMX management application and the platform MBeanServer can interoperate without requiring classes for MXBean specific data types. The data types being transmitted between the JMX connector server and the connector client are open types and this allows interoperation across versions. See the specification of MXBeans for details.

Each platform MXBean is a PlatformManagedObject and it has a unique ObjectName for registration in the platform MBeanServer as returned by the getObjectName method.

As developer before running or while running any bulk process or schedulers, it’s always better to get system information.

ThrottledTaskRunnerStats Service:

Create ThrottledTaskRunnerStats service as shown below:

package com.aem.operations.core.services;

import javax.management.InstanceNotFoundException;
import javax.management.ReflectionException;

/**
 * Private interface for exposing ThrottledTaskRunner stats
 * **/
public interface ThrottledTaskRunnerStats {
    /**
     * @return the % of CPU being utilized.
     * @throws InstanceNotFoundException
     * @throws ReflectionException
     */
    double getCpuLevel() throws InstanceNotFoundException, ReflectionException;

    /**
     * The % of memory being utilized.
     * @return
     */
    double getMemoryUsage();

    /***
     * @return the OSGi configured max allowed CPU utilization.
     */
    double getMaxCpu();

    /***
     * @return the OSGi configured max allowed Memory (heap) utilization.
     */
    double getMaxHeap();

    /**
     * @return the max number of threads ThrottledTaskRunner will use to execute the work.
     */
    int getMaxThreads();
}

ThrottledTaskRunnerImpl Service Implementation:

Create ThrottledTaskRunnerImpl service implementation as shown below:

package com.aem.operations.core.services.impl;

import java.lang.management.ManagementFactory;
import javax.management.Attribute;
import javax.management.AttributeList;
import javax.management.AttributeNotFoundException;
import javax.management.InstanceNotFoundException;
import javax.management.MBeanException;
import javax.management.MBeanServer;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;
import javax.management.ReflectionException;
import javax.management.openmbean.CompositeData;
import com.aem.operations.core.services.ThrottledTaskRunnerStats;
import org.osgi.service.component.annotations.Activate;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Modified;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Component(service = ThrottledTaskRunnerStats.class, immediate = true, name = "Throttled Task Runner Service Stats")
public class ThrottledTaskRunnerImpl implements ThrottledTaskRunnerStats {

    private static final Logger LOGGER = LoggerFactory.getLogger(ThrottledTaskRunnerImpl.class);
    private final MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
    private ObjectName osBeanName;
    private ObjectName memBeanName;

    @Activate
    @Modified
    protected void activate() {
        try {
            memBeanName = ObjectName.getInstance("java.lang:type=Memory");
            osBeanName = ObjectName.getInstance("java.lang:type=OperatingSystem");
        } catch (MalformedObjectNameException | NullPointerException ex) {
            LOGGER.error("Error getting OS MBean (shouldn't ever happen) {}", ex.getMessage());
        }
    }

    @Override
    public double getCpuLevel() throws InstanceNotFoundException, ReflectionException {
        // This method will block until CPU usage is low enough
        AttributeList list = mbs.getAttributes(osBeanName, new String[]{"ProcessCpuLoad"});

        if (list.isEmpty()) {
            LOGGER.error("No CPU stats found for ProcessCpuLoad");
            return -1;
        }

        Attribute att = (Attribute) list.get(0);
        return (Double) att.getValue();
    }

    @Override
    public double getMemoryUsage() {
        try {
            Object memoryusage = mbs.getAttribute(memBeanName, "HeapMemoryUsage");
            CompositeData cd = (CompositeData) memoryusage;
            long max = (Long) cd.get("max");
            long used = (Long) cd.get("used");
            return (double) used / (double) max;
        } catch (AttributeNotFoundException | InstanceNotFoundException | MBeanException | ReflectionException e) {
            LOGGER.error("No Memory stats found for HeapMemoryUsage", e);
            return -1;
        }
    }

    @Override
    public double getMaxCpu() {
        return 0.75;
    }

    @Override
    public double getMaxHeap() {
        return 0.85;
    }

    @Override
    public int getMaxThreads() {
        return Math.max(1, Runtime.getRuntime().availableProcessors()/2);
    }
}

CPU Status Servlet:

Create a CpuStatusServlet based on the path as shown below: 

package com.aem.operations.core.servlets;

import java.io.IOException;
import java.io.PrintWriter;
import java.text.MessageFormat;
import javax.management.InstanceNotFoundException;
import javax.management.ReflectionException;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import com.aem.operations.core.services.ThrottledTaskRunnerStats;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.gson.Gson;
import com.google.gson.JsonObject;

@Component(service = { Servlet.class }, property = { "sling.servlet.paths=" + CpuStatusServlet.RESOURCE_PATH,
        "sling.servlet.methods=POST" })
public class CpuStatusServlet extends SlingAllMethodsServlet{

    private static final Logger LOGGER = LoggerFactory.getLogger(CpuStatusServlet.class);

    private static final long serialVersionUID = 1L;
    public static final String RESOURCE_PATH = "/bin/cpustatust";
    private static final String MESSAGE_FORMAT = "{0,number,#%}";

    @Reference
    private transient ThrottledTaskRunnerStats ttrs;

    @Override
    protected void doGet(SlingHttpServletRequest request, SlingHttpServletResponse response)
            throws ServletException, IOException {
        JsonObject jsonResponse = getSystemStats();
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }

    private JsonObject getSystemStats() {
        JsonObject json = new JsonObject();
        try {
            json.addProperty("cpu", MessageFormat.format(MESSAGE_FORMAT, ttrs.getCpuLevel()));
        } catch (InstanceNotFoundException | ReflectionException e) {
            LOGGER.error("Could not collect CPU stats {}", e.getMessage());
            json.addProperty("cpu", -1);
        }
        json.addProperty("mem", MessageFormat.format(MESSAGE_FORMAT, ttrs.getMemoryUsage()));
        json.addProperty("maxCpu", MessageFormat.format(MESSAGE_FORMAT, ttrs.getMaxCpu()));
        json.addProperty("maxMem", MessageFormat.format(MESSAGE_FORMAT, ttrs.getMaxHeap()));
        return json;
    }
}

Frontend:

Make a request to a servlet from frontend using AJAX call as shown below:

function getStatus(showStatus) {
   $.ajax({
      // add the servlet path
      url: "/bin/cpustatust",
      method: "GET",
      async: true,
      cache: false,
      contentType: false,
      processData: false
   }).done(function (data) {
      if (data) {
         data = JSON.parse(data);
         $("#table-body").append(`<tr>
						<td>${data.cpu}/${data.maxCpu}</td>
			            <td>${data.mem}/${data.maxMem}</td>
		           		 </tr>`);
      } else {
         $(".modal").hide();
         showStatus = false;
         ui.notify("Error", "Unable to get the status", "error");
      }
   }).fail(function (data) {
      $(".modal").hide();
      showStatus = false;
      if (data && data.responseJSON && data.responseJSON.message) {
         ui.notify("Error", data.responseJSON.message, "error");
      } else {
         //add error message
         ui.notify("Error", "Unable to get the status", "error");
      }
   });
   if (showStatus) {
      setTimeout(() => {
         emptyResults();
         getStatus(true);
      }, 2000);
   }
}
function emptyResults() {
   $("#table-body").empty();
}
<table id="table-data">
   <thead id="table-head">
   <tr>
      <th>CPU Usage</th>
      <th>Memory (Heap) Usage</th>
   </tr>
   </thead>
   <tbody id="table-body">
   </tbody>
</table>

The output of the status:

CPU Memory status result

For more information on Throttled Task runner-based performance optimization please visit:

  1. AEM Performance Optimization Scheduler
  2. AEM Performance Optimization Workflow
  3. AEM Performance Optimization Replication

AEM Performance Optimization Activation/Replication

Problem Statement:

AEM Bulk Replication allows you to activate a series of pages and/or assets.

How can we make sure workflow won’t impact AEM performance (CPU or Heap memory) / throttle the system?

Introduction:

AEM bulk replication or activation is performed on a series of pages and/or assets. We usually perform bulk replication on tree structures or lists of paths.

Use case:

  1. MSM (Multi-site management) – rolling out a series of pages or site
  2. Editable template – add/remove new components on template structure and activate existing pages
  3. Bulk Asset ingests into the system
  4. Bulk redirect//vanity path update

ACS Commons Throttled Task Runner is built on Java management API for managing and monitoring the Java VM and can be used to pause tasks and terminate the tasks based on stats.

Throttled Task Runner (a managed thread pool) provides a convenient way to run many AEM-specific activities in bulk it basically checks for the Throttled Task Runner bean and gets current running stats of the actual work being done.

OSGi Configuration:

The Throttled Task Runner is OSGi configurable, but please note that changing configuration while work is being processed results in resetting the worker pool and can lose active work.

Throttled Task Runner OSGi

Max threads: Recommended not to exceed the number of CPU cores. Default 4.

Max CPU %: Used to throttle activity when CPU exceeds this amount. Range is 0..1; -1 means disable this check.

Max Heap %: Used to throttle activity when heap usage exceeds this amount. Range is 0..1; -1 means disable this check.

Cooldown time: Time to wait for CPU/MEM cooldown between throttle checks (in milliseconds)

Watchdog time: Maximum time allowed (in ms) per action before it is interrupted forcefully.

JMX MBeans

Throttled Task Runner MBean

This is the core worker pool. All action managers share the same task runner pool, at least in the current implementation. The task runner can be paused or halted entirely, throwing out any unfinished work.

Throttled task runner JMX
Throttled task runner JMX

How to use ACS Commons throttled task runner

Add the following dependency to your pom

Create a custom service or servlet as shown below:

Throttled Replication

inside the custom workflow process method check for the Low CPU and Low memory before starting your task to avoid performance impact on the system.

For bulk replication (publish/unpublish/delete) assets or pages, please refer to the AEM Operation Replication Tool

For best practices on the AEM servlet please refer to the link.

AEM Performance Optimization Workflow

Problem Statement:

AEM Workflows allow you to automate a series of steps that are performed on (one or more) pages and/or assets.

How can we make sure workflow won’t impact AEM performance (CPU or Heap memory) / throttle the system?

Introduction:

AEM Workflows allow you to automate a series of steps that are performed on (one or more) pages and/or assets.

For example, when publishing, an editor has to review the content – before a site administrator activates the page. A workflow that automates this example notifies each participant when it is time to perform their required work:

  1. The author applies the workflow to the page.
  2. The editor receives a work item that indicates that they are required to review the page content. When finished, they indicate that their work item is complete.
  3. The site administrator then receives a work item that requests the activation of the page. When finished, they indicate that their work item is complete.

Typically:

  • Content authors apply workflows to pages as well as participate in workflows.
  • The workflows that you use are specific to the business processes of your organization.

ACS Commons Throttled Task Runner is built on Java management API for managing and monitoring the Java VM and can be used to pause tasks and terminate the tasks based on stats.

Throttled Task Runner (a managed thread pool) provides a convenient way to run many AEM-specific activities in bulk it basically checks for the Throttled Task Runner bean and gets current running stats of the actual work being done.

OSGi Configuration:

The Throttled Task Runner is OSGi configurable, but please note that changing configuration while work is being processed results in resetting the worker pool and can lose active work.

Throttled task runner OSGi

Max threads: Recommended not to exceed the number of CPU cores. Default 4.

Max CPU %: Used to throttle activity when CPU exceeds this amount. Range is 0..1; -1 means disable this check.

Max Heap %: Used to throttle activity when heap usage exceeds this amount. Range is 0..1; -1 means disable this check.

Cooldown time: Time to wait for CPU/MEM cooldown between throttle checks (in milliseconds)

Watchdog time: Maximum time allowed (in ms) per action before it is interrupted forcefully.

JMX MBeans

Throttled Task Runner MBean

This is the core worker pool. All action managers share the same task runner pool, at least in the current implementation. The task runner can be paused or halted entirely, throwing out any unfinished work.

Throttled task runner JMX
Throttled task runner JMX

How to use ACS Commons throttled task runner

Add the following dependency to your pom

<dependency>
    <groupId>com.adobe.acs</groupId>
    <artifactId>acs-aem-commons-bundle</artifactId>
    <version>5.0.4</version>
    <scope>provided</scope>
</dependency>

Create a custom workflow and call the service as shown below:

Throttled Workflow

inside the custom workflow process method check for the Low CPU and Low memory before starting your task to avoid performance impact on the system.

For all the bulk workflow executions on pages/assets, it’s recommended to use ACS commons bulk workflow.

For best practices on workflow please refer to the link

AEM Performance Optimization Scheduler

Problem Statement:

AEM Schedulers are commonly used to run bulk tasks at an off time (nonbusiness hours) and some tasks are run periodically to fetch results cached and fetched by the front end.

How can we make sure bulk tasks won’t impact AEM performance (CPU or Heap memory) / throttle the system?

Introduction:

In Computer Software, Scheduling is a paradigm of planning for the execution of a task at a certain point in time and it can be broadly classified into two types:

1. Scheduled Task – executing once at a particular future point in time

2. Frequent scheduling – repeat periodically at a fixed interval

The use case for the scheduler:

1. Sitemap generation

2. Synching product data from AEM Commerce

3. Fetch DB content and place it in the repository to be picked up by frontend and in turn cached in Dispatcher

4. Fetch Stats or reports and place them in the repository to be picked up by frontend and in turn cached in Dispatcher

ACS Commons Throttled Task Runner is built on Java management API for managing and monitoring the Java VM and can be used to pause tasks and terminate the tasks based on stats.

Throttled Task Runner (a managed thread pool) provides a convenient way to run many AEM-specific activities in bulk it basically checks for the Throttled Task Runner bean and gets current running stats of the actual work being done.

OSGi Configuration:

The Throttled Task Runner is OSGi configurable, but please note that changing configuration while work is being processed results in resetting the worker pool and can lose active work.

Throttled task runner OSGi

Max threads: Recommended not to exceed the number of CPU cores. Default 4.

Max CPU %: Used to throttle activity when CPU exceeds this amount. Range is 0..1; -1 means disable this check.

Max Heap %: Used to throttle activity when heap usage exceeds this amount. Range is 0..1; -1 means disable this check.

Cooldown time: Time to wait for CPU/MEM cooldown between throttle checks (in milliseconds)

Watchdog time: Maximum time allowed (in ms) per action before it is interrupted forcefully.

JMX MBeans

Throttled Task Runner MBean

This is the core worker pool. All action managers share the same task runner pool, at least in the current implementation. The task runner can be paused or halted entirely, throwing out any unfinished work.

Throttled task runner JMX

How to use ACS Commons throttled task runner

Add the following dependency to your pom

<dependency>
     <groupId>com.adobe.acs</groupId>
     <artifactId>acs-aem-commons-bundle</artifactId>
     <version>5.0.4</version>
     <scope>provided</scope>
 </dependency>

Create a scheduler and call the service as shown below:

Throttled Scheduler call

inside the run, method check for the Low CPU and Low memory before starting your task to avoid performance impact on the system

For the best practices on AEM Scheduler please refer to the link

AEM Tool – Access CRX Package manager in AEM PROD

Problem statement:

How to access the CRX package manager in AEM PROD or AEM as Cloud services?

User cases:

  1. Latest content package from PROD to lowers or local for debugging purposes
  2. Install the content package on PROD
  3. Continue the PROD deployment during CM outage in between deployment

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

  1. AEM operations
  2. Managing templates
  3. Cloud configurations
  4. ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Package handler can be used to upload, install, build or delete packages and we are using JCR Package manager to achieve all the above options.

Usually, if want to perform any operations on AEM as managed services are AEMaaCS we need to go through CAB, and AMS resources will perform all the operations as mentioned on the CAB. However, if your project has shared resources then all the priority package operations will take more time on PROD or any other environments.

In order to create a perfect content backup package please use the Tool generator to give your tool name and descriptions: https://kiransg.com/2022/11/24/aem-tool-create-generate-tool-from-scratch

Add the following Sling model fields as shown below to accept the inputs:

package com.aem.operations.core.models;

import java.io.InputStream;
import javax.inject.Named;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import com.adobe.acs.commons.mcp.form.FileUploadComponent;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.GeneratedDialog;
import com.adobe.acs.commons.mcp.form.RadioComponent;
import lombok.Getter;
import lombok.Setter;

@Model(adaptables = { Resource.class,SlingHttpServletRequest.class }, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class PackageHandlerModel extends GeneratedDialog {

    @Getter
    @Setter
    @Named(value = "inputPackage")
    @FormField(name = "Package", description = "Upload JCR Package", component = FileUploadComponent.class)
    private InputStream inputPackage = null;

    @Getter
    @Setter
    @Named(value = "packageName")
    @FormField(
            name = "Package Name",
            category = "General",
            required = false,
            description = "Enter the package name",
            hint = "Name")
    private String packageName;

    @Getter
    @Setter
    @Named(value = "packageGroup")
    @FormField(
            name = "Package Group",
            category = "General",
            required = false,
            description = "Enter the package group",
            hint = "Group")
    private String packageGroup;

    @Getter
    @Setter
    @Named(value = "packageVersion")
    @FormField(
            name = "Package Version",
            category = "General",
            required = false,
            description = "Enter the package version",
            hint = "Version")
    private String packageVersion;

    @Getter
    @Setter
    @Named(value = "packageOperation")
    @FormField(
            name = "Package Operation",
            description = "Select the operation to be performed",
            required = true,
            component = RadioComponent.EnumerationSelector.class,
            options = {"horizontal", "default=UPLOAD"})
    private Mode packageOperation;

    public enum Mode {
        UPLOAD, UPLOAD_INSTALL, BUILD, INSTALL, DELETE
    }
}

Add the following Servlet to process the request as shown below:

package com.aem.operations.core.servlets;

import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import com.aem.operations.core.models.PackageHandlerModel;
import com.aem.operations.core.services.PackageHandlerService;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.sling.servlets.post.JSONResponse;
import org.jetbrains.annotations.Nullable;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.granite.rest.Constants;
import com.google.gson.Gson;
import com.google.gson.JsonObject;

@Component(
        service = { Servlet.class },
        property = {
                "sling.servlet.paths=" + PackageHandlerServlet.RESOURCE_PATH,
                "sling.servlet.methods=POST"
        }
)
public class PackageHandlerServlet extends SlingAllMethodsServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger LOGGER = LoggerFactory.getLogger(PackageHandlerServlet.class);
    public static final String RESOURCE_PATH = "/bin/triggerPackageHandler";
    private static final String MESSAGE = "message";

    @Reference
    private PackageHandlerService packageHandlerService;

    @Override
    protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response) throws ServletException, IOException {
        // Set response headers.
        response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE);
        response.setCharacterEncoding(Constants.DEFAULT_CHARSET);

        JsonObject jsonResponse = new JsonObject();
        ResourceResolver resourceResolver = request.getResourceResolver();

        String packageName = request.getParameter("packageName");
        String packageGroup = request.getParameter("packageGroup");
        String packageVersion = request.getParameter("packageVersion");
        String packageOperation = request.getParameter("packageOperation");
        @Nullable RequestParameter inputPackStream = request.getRequestParameter("file");

        String packagePath = StringUtils.EMPTY;
        if (StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.UPLOAD.toString(), packageOperation)) {
            packagePath = packageHandlerService.uploadPack(resourceResolver, inputPackStream);
        } else  if(StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.BUILD.toString(), packageOperation)) {
            packagePath = packageHandlerService.buildPackage(resourceResolver, packageGroup, packageName, packageVersion);
        } else if(StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.UPLOAD_INSTALL.toString(), packageOperation)) {
            packagePath = packageHandlerService.uploadAndInstallPack(resourceResolver, inputPackStream);
        } else if(StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.INSTALL.toString(), packageOperation)) {
            packagePath = packageHandlerService.installPackage(resourceResolver, packageGroup, packageName, packageVersion, ImportMode.REPLACE, AccessControlHandling.IGNORE);
        } else if(StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.DELETE.toString(), packageOperation)) {
            packagePath = packageHandlerService.deletePackage(resourceResolver, packageGroup, packageName, packageVersion);
        }
        if(StringUtils.isNotEmpty(packagePath)) {
            jsonResponse.addProperty(MESSAGE, "Package Operation is Successful and path is <a href=\"" + packagePath + "\">" + packagePath + "</a>");
        } else {
            LOGGER.error("Unable to process package operation");
        }
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }
}

Add the following Package service as shown below:

package com.aem.operations.core.services;

import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.ResourceResolver;
import org.jetbrains.annotations.Nullable;

public interface PackageHandlerService {

	
	/**
	 * @param resourceResolver
	 * @param inputPackage
	 * @return package Path
	 */
	public String uploadPack(ResourceResolver resourceResolver, @Nullable RequestParameter inputPackage);

	/**
	 * @param resourceResolver
	 * @param groupName
	 * @param packageName
	 * @param version
	 * @return
	 */
	public String buildPackage(ResourceResolver resourceResolver, final String groupName, final String packageName,
			final String version);

	/**
	 * @param resourceResolver
	 * @param groupName
	 * @param packageName
	 * @param version
	 * @param importMode
	 * @param aclHandling
	 * @return package Path
	 */
	public String installPackage(ResourceResolver resourceResolver, final String groupName, final String packageName,
			final String version, final ImportMode importMode, final AccessControlHandling aclHandling);

	/**
	 * @param resourceResolver
	 * @param inputPackage
	 * @return package Path
	 */
	public String uploadAndInstallPack(ResourceResolver resourceResolver, @Nullable RequestParameter inputPackage);

	/**
	 * @param resourceResolver
	 * @param groupName
	 * @param packageName
	 * @param version
	 * @return package Path
	 */
	public String deletePackage(ResourceResolver resourceResolver, final String groupName, final String packageName,
			final String version);
}

Add the following Package service Implementation as shown below:

package com.aem.operations.core.services.impl;

import com.aem.operations.core.services.PackageHandlerService;
import com.aem.operations.core.utils.VltUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.fs.io.ImportOptions;
import org.apache.jackrabbit.vault.packaging.*;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.ResourceResolver;
import org.jetbrains.annotations.Nullable;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import java.io.IOException;
import java.io.InputStream;

@Component(service = PackageHandlerService.class, immediate = true, name = "Package Handler Service")
public class PackageHandlerServiceImpl implements PackageHandlerService {

	private static final Logger LOGGER = LoggerFactory.getLogger(PackageHandlerServiceImpl.class);

	@Reference
	private Packaging packaging;

	@Override
	public String uploadPack(ResourceResolver resourceResolver, @Nullable RequestParameter inputPackage) {
		if (null != inputPackage) {
			try (InputStream inputPack = inputPackage.getInputStream()) {
				if (null != inputPack) {
					final JcrPackageManager packageManager = packaging
							.getPackageManager(resourceResolver.adaptTo(Session.class));
					JcrPackage jcrPackage = packageManager.upload(inputPack, true);
					return jcrPackage.getNode().getPath();
				}
			} catch (IOException | RepositoryException e) {
				LOGGER.error("Could not upload package {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

	@Override
	public String buildPackage(ResourceResolver resourceResolver, String groupName, String packageName,
			String version) {
		if (StringUtils.isNotEmpty(packageName) && StringUtils.isNotEmpty(groupName)) {
			final JcrPackageManager packageManager = packaging
					.getPackageManager(resourceResolver.adaptTo(Session.class));
			final PackageId packageId = new PackageId(groupName, packageName, version);
			try (JcrPackage jcrPackage = packageManager.open(packageId)) {
				if(null != jcrPackage) {
					packageManager.assemble(jcrPackage, null);
					return jcrPackage.getNode().getPath();
				}
			} catch (RepositoryException | PackageException | IOException e) {
				LOGGER.error("Could not build package {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

	@Override
	public String installPackage(ResourceResolver resourceResolver, String groupName, String packageName,
			String version, ImportMode importMode, AccessControlHandling aclHandling) {
		if (StringUtils.isNotEmpty(packageName) && StringUtils.isNotEmpty(groupName)) {
			final JcrPackageManager packageManager = packaging
					.getPackageManager(resourceResolver.adaptTo(Session.class));
			final PackageId packageId = new PackageId(groupName, packageName, version);
			try (JcrPackage jcrPackage = packageManager.open(packageId)) {
				if(null != jcrPackage) {
					final ImportOptions opts = VltUtils.getImportOptions(aclHandling, importMode);
					jcrPackage.install(opts);
					return jcrPackage.getNode().getPath();
				}
			} catch (RepositoryException | PackageException | IOException e) {
				LOGGER.error("Could not install package {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

	@Override
	public String uploadAndInstallPack(ResourceResolver resourceResolver, @Nullable RequestParameter inputPackage) {
		if (null != inputPackage) {
			try (InputStream inputPack = inputPackage.getInputStream()) {
				if (null != inputPack) {
					final JcrPackageManager packageManager = packaging
							.getPackageManager(resourceResolver.adaptTo(Session.class));
					try (JcrPackage jcrPackage = packageManager.upload(inputPack, true)) {
						installPackage(jcrPackage, ImportMode.REPLACE, AccessControlHandling.IGNORE);
						return jcrPackage.getNode().getPath();
					} catch (RepositoryException e) {
						LOGGER.error("Could not Upload and Install package due to Repository Exception {}", e.getMessage());
						e.printStackTrace();
					}
				}
			} catch (IOException e) {
				LOGGER.error("Could not Upload and Install package due to IO Exception {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

	public String installPackage(JcrPackage jcrPackage, final ImportMode importMode,
			final AccessControlHandling aclHandling) {
		try {
			final ImportOptions opts = VltUtils.getImportOptions(aclHandling, importMode);
			jcrPackage.install(opts);
			return jcrPackage.getNode().getPath();
		} catch (RepositoryException | PackageException | IOException e) {
			LOGGER.error("Could not install built package {}", e.getMessage());
			return StringUtils.EMPTY;
		}
	}

	@Override
	public String deletePackage(ResourceResolver resourceResolver, String groupName, String packageName,
			String version) {
		if (StringUtils.isNotEmpty(packageName) && StringUtils.isNotEmpty(groupName)) {
			final JcrPackageManager packageManager = packaging
					.getPackageManager(resourceResolver.adaptTo(Session.class));
			final PackageId packageId = new PackageId(groupName, packageName, version);
			try (JcrPackage jcrPackage = packageManager.open(packageId)) {
				if(null != jcrPackage) {
					String path = jcrPackage.getNode().getPath();
					packageManager.remove(jcrPackage);
					return path;
				}				
			} catch (RepositoryException e) {
				LOGGER.error("Could not delete package {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

}

Add the following VltUtils as shown below:

package com.aem.operations.core.utils;

import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.fs.io.ImportOptions;

/**
 * Utility class for creating vlt filters and import/export options
 */
public class VltUtils {

    public static ImportOptions getImportOptions(AccessControlHandling aclHandling, ImportMode importMode) {
        ImportOptions opts = new ImportOptions();
        if (aclHandling != null) {
            opts.setAccessControlHandling(aclHandling);
        } else {
            // default to overwrite
            opts.setAccessControlHandling(AccessControlHandling.OVERWRITE);
        }
        if (importMode != null) {
            opts.setImportMode(importMode);
        } else {
            // default to update
            opts.setImportMode(ImportMode.UPDATE);
        }

        return opts;
    }
}

Add the following HTL to render the page:

<!DOCTYPE html>
<head>
   <link rel="shortcut icon" href="/libs/granite/core/content/login/favicon.ico">
</head>
<body class="coral--light foundation-layout-util-maximized-alt">
   <sly data-sly-call="${clientLib.all @ categories=['coralui3','granite.ui.coral.foundation','granite.ui.shell','cq.authoring.dialog', 'packagehandelr.base']}"
      data-sly-use.clientLib="/libs/granite/sightly/templates/clientlib.html"/>
   <coral-shell-header aria-hidden="false" aria-label="Header Bar"
      class="coral--dark granite-shell-header coral3-Shell-header"
      role="region">
      <coral-shell-header-home aria-level="2" class="globalnav-toggle" data-globalnav-toggle-href="/" role="heading">
         <a class="coral3-Shell-homeAnchor" href="/"
            icon="adobeExperienceManagerColor" is="coral-shell-homeanchor"
            style="display: inline-block; padding-right: 0;">
            <coral-icon
               aria-label="adobe experience manager color"
               class="coral3-Icon coral3-Shell-homeAnchor-icon coral3-Icon--adobeExperienceManagerColor coral3-Icon--sizeM"
               icon="adobeExperienceManagerColor" role="img"
               size="M"></coral-icon>
            <coral-shell-homeanchor-label>Adobe Experience Manager
            </coral-shell-homeanchor-label>
         </a>
         <span style="line-height: 2.375rem;">/ AEM Operations / ${properties.jcr:title} </span>
      </coral-shell-header-home>
   </coral-shell-header>
   <div class="foundation-layout-panel">
      <div class="foundation-layout-panel-header">
         <betty-titlebar>
            <betty-titlebar-title>
               <span aria-level="1" class="granite-title" role="heading">${properties.jcr:title}</span>
            </betty-titlebar-title>
            <betty-titlebar-primary></betty-titlebar-primary>
            <betty-titlebar-secondary>
            </betty-titlebar-secondary>
         </betty-titlebar>
      </div>
      <div class="content-container-inner coral-Well" style="width:60%; margin:0 auto;">
         <sly data-sly-use.packprops="com.aem.operations.core.models.PackageHandlerModel">
            <sly data-sly-resource="${packprops.formResource}"/>
         </sly>
         <coral-actionbar-item class="coral3-ActionBar-item">
            <button class="coral3-Button coral3-Button--primary package-initiator" icon="checkCircle" iconsize="S"
               is="coral-button"
               size="M" variant="primary">
               <coral-icon alt="" class="coral3-Icon coral3-Icon--sizeS coral3-Icon--checkCircle" icon="engagement"
                  size="S"></coral-icon>
               <coral-button-label>
                  Start the Process
               </coral-button-label>
            </button>
         </coral-actionbar-item>
          <!-- The Modal -->
          <div id="popup-modal" class="modal">
              <!-- Modal content -->
              <div class="modal-content">
                  <span class="close">&times;</span>
                  <span class="loading">Loading</span>
                  <span class="loader"></span>
              </div>
          </div>
      </div>
   </div>
</body>

Add the following JS code to process the ajax request as shown below:

var Coral = window.Coral || {},
    Granite = window.Granite || {};

(function (window, document, $, Coral) {
    "use strict";
    var packageOperation = "UPLOAD";
    $(document).on("foundation-contentloaded", function (e) {

        var SITE_PATH = "/conf/aemoperations/settings/tools/packagehandler-initiator.html",
            ui = $(window).adaptTo("foundation-ui");

        if (window.location.href.indexOf(SITE_PATH) < 0) {
            return;
        }

        if (packageOperation === "UPLOAD") {
            $("input[name='./packageName']").attr("disabled", true);
            $("input[name='./packageGroup']").attr("disabled", true);
            $("input[name='./packageVersion']").attr("disabled", true);
        }

        $(document).off("change", ".coral-RadioGroup").on("change", ".coral-RadioGroup", function (event) {
            packageOperation = event.target.value;
			if (packageOperation === "UPLOAD" || packageOperation === "UPLOAD_INSTALL") {
				$("input[name='./packageName']").attr("disabled", true);
				$("input[name='./packageGroup']").attr("disabled", true);
				$("input[name='./packageVersion']").attr("disabled", true);
				$("coral-fileupload[name='./inputPackage']").attr("disabled", false);
			} else if(packageOperation === "BUILD" || packageOperation === "INSTALL" || packageOperation === "DELETE"){
				$("input[name='./packageName']").attr("disabled", false);
				$("input[name='./packageGroup']").attr("disabled", false);
				$("input[name='./packageVersion']").attr("disabled", false);
				$("coral-fileupload[name='./inputPackage']").attr("disabled", true);
			}
        });

        $(document).off("change", ".close").on("change", ".close", function (event) {
            $(".modal").hide();
        });

        function getValueByName(fieldName, isMandatory) {
            var fieldValue = ($("input[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("input[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("input[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getFileByName(fieldName) {
            var fieldInput = $("input[name='" + fieldName + "']");
            if(null != fieldInput && null != fieldInput[0]){
                return fieldInput[0].files[0];
            }
        }

        $(document).off("click", ".package-initiator").on("click", ".package-initiator", function (event) {
            event.preventDefault();

            var packageName = getValueByName('./packageName', false),
                packageGroup = getValueByName('./packageGroup', false),
                packageVersion = getValueByName('./packageVersion', false),
                inputPackage = getFileByName('./inputPackage');

            if (!packageOperation) {
                return;
            }

            var formData = new FormData();
            if(null != inputPackage){
                formData.append("file", inputPackage, inputPackage.name);
            }
            formData.append("packageName", packageName);
            formData.append("packageGroup", packageGroup);
            formData.append("packageVersion", packageVersion);
            formData.append("packageOperation", packageOperation);

			$(".loading").html("PLEASE WAIT "+packageOperation+"ING PACKAGE");
			$(".modal").show();

            $.ajax({
                url: "/bin/triggerPackageHandler",
                method: "POST",
                async: true,
                cache: false,
                contentType: false,
                processData: false,
                data: formData
            }).done(function (data) {
                if (data && data.message){
                    ui.notify("Success", data.message, "success");
                    var dialog = new Coral.Dialog();
                    dialog.id = 'dialogSuccess';
                    dialog.header.innerHTML = 'Success';
                    dialog.content.innerHTML = data.message;
                    dialog.footer.innerHTML = '<button class="ok-button" is="coral-button" variant="primary" icon="check" coral-close>OK</button>';
                    dialog.variant = 'success';
                    dialog.closable = "on";
                    dialog.show();
					$(".modal").hide();
                }else{
                    ui.notify("Error", "Unable to process package operation", "error");
                }
            }).fail(function (data) {
				$(".modal").hide();
                if (data && data.responseJSON && data.responseJSON.message){
                    ui.notify("Error", data.responseJSON.message, "error");
                }else{
                    ui.notify("Error", "Unable to process package operation", "error");
                }
            });
        });
    });
})(window, document, $, Coral);

Once the code is built and deployed you will be able to access the tool by navigating the:

Tools -> {Tool Section} -> {Tool Name}

Package Handler Tool Section

Now you can access the package handler and upload the packages to AEM as an upload or upload and install.

Package Handler Page

If you’re building any existing packages then you can build, install or delete packages. Provide package name, version, and group details to pull the package.

Provide the desired package name and Description and you can see the package path will be prompted and you will be able to download the built package by clicking on the link:

Package Handler Result
test package is uploaded

Working Code:
You can also access the working code on my GitHub repository link: https://github.com/kiransg89/aemoperations

Note: You can access this package over domain on AEM / AEMaaCS and over any environments and architects can handle the permissions by adding appropriate rep:policy on cq generator node or conf page

AEM Tool – Publish / UnPublish / Delete List of pages

Problem statement:

Can we Publish / Un Publish / Delete the list of paths mentioned in an excel sheet? Or provided has a linefeed

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

  1. AEM operations
  2. Managing templates
  3. Cloud configurations
  4. ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Usually, product owners or authors would like to Publish certain pages like offer/product/article pages based on business requirements and also would like to unpublish and delete to clean up the pages which are unnecessary.

This process will be really helpful during excel sheet based migration.

The list Replication process usually validates the paths and if it exists then activates or deactivates or deletes pages or content paths. It accepts an excel sheet or list of paths as a line feed.

You can also provide a list of agents to be activated/deactivated to for example only to Brightspot or Brightcove connector.

In order to create a tool please use the Tool generator to give your tool name and description: https://kiransg.com/2022/11/24/aem-tool-create-generate-tool-from-scratch

Add the following Sling model fields as shown below to accept the inputs:

package com.aem.operations.core.models;

import com.adobe.acs.commons.mcp.form.*;
import lombok.Getter;
import lombok.Setter;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import javax.inject.Named;
import java.io.InputStream;

@Model(adaptables = { Resource.class,SlingHttpServletRequest.class }, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class ListTreeActivationModel extends GeneratedDialog {

    public enum QueueMethod {
        USE_MCP_QUEUE, USE_PUBLISH_QUEUE, MCP_AFTER_10K
    }

    public enum ReplicationAction {
        PUBLISH, UNPUBLISH, DELETE
    }

    @Getter
    @Setter
    @Named(value = "repPathExcel")
    @FormField(name = "Replication Excel", description = "Upload Replication Excel", component = FileUploadComponent.class)
    private InputStream repPathExcel = null;

    @Getter
    @Setter
    @Named(value = "queueMethod")
    @FormField(
            name = "Queueing Method",
            description = "For small publishing tasks, standard is sufficient.  For large folder trees, MCP is recommended.",
            required = true,
            component = SelectComponent.EnumerationSelector.class,
            options = {"horizontal", "default=USE_MCP_QUEUE"})
    QueueMethod queueMethod = QueueMethod.USE_MCP_QUEUE;

    @Getter
    @Setter
    @Named(value = "agents")
    @FormField(name = "Agents",
            component = TextfieldComponent.class,
            hint = "(leave blank for default agents)",
            description = "Publish agents to use, if blank then all default agents will be used. Multiple agents can be listed using commas or regex.")
    private String agents;

    @Getter
    @Setter
    @Named(value = "reAction")
    @FormField(name = "Action",
            component = SelectComponent.EnumerationSelector.class,
            description = "Publish or Unpublish?",
            options = "default=PUBLISH")
    ReplicationAction reAction = ReplicationAction.PUBLISH;
}

Add the following Servlet to process the request as shown below:

package com.aem.operations.core.servlets;

import java.io.IOException;
import java.io.PrintWriter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
import javax.jcr.Session;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import com.aem.operations.core.models.ListTreeActivationModel;
import com.aem.operations.core.utils.RetryUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.sling.servlets.post.JSONResponse;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.acs.commons.data.CompositeVariant;
import com.adobe.acs.commons.data.Spreadsheet;
import com.adobe.granite.rest.Constants;
import com.day.cq.replication.AgentFilter;
import com.day.cq.replication.ReplicationActionType;
import com.day.cq.replication.ReplicationException;
import com.day.cq.replication.ReplicationOptions;
import com.day.cq.replication.Replicator;
import com.google.gson.Gson;
import com.google.gson.JsonObject;

@Component(service = { Servlet.class }, property = { "sling.servlet.paths=" + ListActiavationServlet.RESOURCE_PATH,
        "sling.servlet.methods=POST" })
public class ListActiavationServlet extends SlingAllMethodsServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger LOGGER = LoggerFactory.getLogger(ListActiavationServlet.class);
    public static final String RESOURCE_PATH = "/bin/triggerListActivation";
    private static final String MESSAGE = "message";
    private static final String DESTINATION_PATH = "destination";
    private static int ASYNC_LIMIT = 10000;


    private List<String> agentList = new ArrayList<>();
    private Spreadsheet spreadsheet;
    AtomicInteger replicationCount = new AtomicInteger();

    @Reference
    Replicator replicator;

    @Override
    protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response)
            throws ServletException, IOException {

        // Set response headers.
        response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE);
        response.setCharacterEncoding(Constants.DEFAULT_CHARSET);

        JsonObject jsonResponse = new JsonObject();

        String queueMethod = request.getParameter("queueMethod");
        String agents = request.getParameter("agents");
        String reAction = request.getParameter("reAction");
        String pathsList = request.getParameter("pathsList");

        @Nullable
        RequestParameter inputPackStream = request.getRequestParameter("file");
        if(inputPackStream != null) {
            spreadsheet = new Spreadsheet(inputPackStream, DESTINATION_PATH).buildSpreadsheet();
        }
        if (null != spreadsheet) {
            AgentFilter replicationAgentFilter = prepareAgents(agents);
            if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.PUBLISH.toString(), reAction)) {
                activateTreeStructure(request.getResourceResolver(), queueMethod, reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Activated all the paths provided in Excel sheet");
            } else if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.UNPUBLISH.toString(), reAction)) {
                deactivateTreeStructure(request.getResourceResolver(), reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deactivated all the paths provided in Excel sheet");
            } else {
                deleteTree(request.getResourceResolver());
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deleted all the paths provided in Excel sheet");
            }
        } else if(StringUtils.isNotEmpty(pathsList)){
            AgentFilter replicationAgentFilter = prepareAgents(agents);
            Set<String> pages = new HashSet<>(Arrays.asList(StringUtils.split(pathsList, '\n'))).stream().map(String::trim).collect(Collectors.toSet());
            if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.PUBLISH.toString(), reAction)) {
                activateTreeStructure(request.getResourceResolver(), pages, queueMethod, reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Activated all the paths provided in Excel sheet");
            } else if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.UNPUBLISH.toString(), reAction)) {
                deactivateTreeStructure(request.getResourceResolver(), pages, reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deactivated all the paths provided in Line Feed");
            } else {
                deleteTree(request.getResourceResolver(), pages);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deleted all the paths provided in Line Feed");
            }
        } else {
            jsonResponse.addProperty(MESSAGE, "Unable to process Activation request because of invalid Inputs");
        }
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }

    private AgentFilter prepareAgents(String agents) {
        AgentFilter replicationAgentFilter;
        if (StringUtils.isEmpty(agents)) {
            replicationAgentFilter = AgentFilter.DEFAULT;
        } else {
            agentList = Arrays.asList(agents.toLowerCase(Locale.ENGLISH).split(","));
            replicationAgentFilter = agent -> agentList.stream()
                    .anyMatch(p -> p.matches(agent.getId().toLowerCase(Locale.ENGLISH)));
        }
        return replicationAgentFilter;
    }

    private void activateTreeStructure(@NotNull ResourceResolver resourceResolver, String queueMethod, String reAction, AgentFilter replicationAgentFilter) {
        spreadsheet.getDataRowsAsCompositeVariants()
                .forEach(row -> performReplication(resourceResolver, getString(row, DESTINATION_PATH), queueMethod, reAction, replicationAgentFilter));
    }

    private void deactivateTreeStructure(@NotNull ResourceResolver resourceResolver, String reAction, AgentFilter replicationAgentFilter) {
        spreadsheet.getDataRowsAsCompositeVariants()
                .forEach(row -> performAsynchronousReplication(resourceResolver, getString(row, DESTINATION_PATH), reAction, replicationAgentFilter));
    }

    private void activateTreeStructure(@NotNull ResourceResolver resourceResolver, Set<String> pages, String queueMethod, String reAction, AgentFilter replicationAgentFilter) {
        pages.stream().forEach(path -> performReplication(resourceResolver, path, queueMethod, reAction, replicationAgentFilter));
    }

    private void deactivateTreeStructure(@NotNull ResourceResolver resourceResolver, Set<String> pages, String reAction, AgentFilter replicationAgentFilter) {
        pages.stream().forEach(path -> performAsynchronousReplication(resourceResolver, path, reAction, replicationAgentFilter));
    }

    private void performReplication(@NotNull ResourceResolver resourceResolver, String path, String queueMethod, String reAction, AgentFilter replicationAgentFilter) {
        int counter = replicationCount.incrementAndGet();
        if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.QueueMethod.USE_MCP_QUEUE.toString(), queueMethod)
                || (StringUtils.equalsIgnoreCase(ListTreeActivationModel.QueueMethod.MCP_AFTER_10K.toString(),
                queueMethod) && counter >= ASYNC_LIMIT)) {
            performSynchronousReplication(resourceResolver, path, reAction, replicationAgentFilter);
        } else {
            performAsynchronousReplication(resourceResolver, path, reAction, replicationAgentFilter);
        }
    }

    private void performSynchronousReplication(@NotNull ResourceResolver resourceResolver, String path, String reAction, AgentFilter replicationAgentFilter) {
        ReplicationOptions options = buildOptions(replicationAgentFilter);
        options.setSynchronous(true);
        scheduleReplication(resourceResolver, options, path, reAction);
    }

    private void performAsynchronousReplication(@NotNull ResourceResolver resourceResolver, String path, String reAction, AgentFilter replicationAgentFilter) {
        ReplicationOptions options = buildOptions(replicationAgentFilter);
        options.setSynchronous(false);
        scheduleReplication(resourceResolver, options, path, reAction);
    }

    private void scheduleReplication(@NotNull ResourceResolver resourceResolver, ReplicationOptions options,
                                     String path, String reAction) {
        Session session = resourceResolver.adaptTo(Session.class);
        boolean isPublish = StringUtils
                .equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.PUBLISH.toString(), reAction);
        try {
            replicator.replicate(session,
                    isPublish ? ReplicationActionType.ACTIVATE : ReplicationActionType.DEACTIVATE, path, options);
        } catch (ReplicationException e) {
            LOGGER.error("Error occured during replication {}", e.getMessage());
        }
    }

    @SuppressWarnings("rawtypes")
    private void deleteTree(ResourceResolver resourceResolver) {
        try {
            RetryUtils.withRetry(30, 500, () -> {
                Iterator<Map<String, CompositeVariant>> spredIterator = spreadsheet.getDataRowsAsCompositeVariants()
                        .iterator();
                while (spredIterator.hasNext()) {
                    Map<String, CompositeVariant> row = spredIterator.next();
                    deleteContent(resourceResolver, getString(row, DESTINATION_PATH));
                }
                commitChanges(resourceResolver);
            });
        } catch (Exception e) {
            LOGGER.error("Error occured during Retrying {}", e.getMessage());
        }
    }

    private void deleteTree(ResourceResolver resourceResolver, Set<String> pages) {
        try {
            RetryUtils.withRetry(30, 500, () -> {
                pages.forEach(path -> deleteContent(resourceResolver, path));
                commitChanges(resourceResolver);
            });
        } catch (Exception e) {
            LOGGER.error("Error occured during Retrying {}", e.getMessage());
        }
    }

    private void deleteContent(ResourceResolver resourceResolver, String destinationPath) {
        try {
            @Nullable
            Resource destinationResource = resourceResolver.getResource(destinationPath);
            if (null != destinationResource) {
                resourceResolver.delete(destinationResource);
            }
        } catch (PersistenceException e) {
            LOGGER.error("unable to delete content {}", e.getMessage());
        }
    }

    private void commitChanges(ResourceResolver resourceResolver) throws PersistenceException {
        if (resourceResolver.hasChanges()) {
            resourceResolver.commit();
            resourceResolver.refresh();
        }
    }

    private ReplicationOptions buildOptions(AgentFilter replicationAgentFilter) {
        ReplicationOptions options = new ReplicationOptions();
        options.setFilter(replicationAgentFilter);
        return options;
    }

    @SuppressWarnings({ "rawtypes", "unchecked" })
    private String getString(Map<String, CompositeVariant> row, String path) {
        CompositeVariant v = row.get(path.toLowerCase(Locale.ENGLISH));
        if (v != null) {
            return (String) v.getValueAs(String.class);
        } else {
            return null;
        }
    }
}

Add the following Retry Utils as shown below:

package com.aem.operations.core.utils;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryUtils {
    private static final Logger log = LoggerFactory.getLogger(RetryUtils.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int i = 0; i < maxTimes; i++) {
            try {
                call.process();
                return true;
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (i + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

Add the following HTL to render the page:

<!DOCTYPE html>
<head>
   <link rel="shortcut icon" href="/libs/granite/core/content/login/favicon.ico">
</head>
<body class="coral--light foundation-layout-util-maximized-alt">
   <sly data-sly-call="${clientLib.all @ categories=['coralui3','granite.ui.coral.foundation','granite.ui.shell','cq.authoring.dialog', 'listreplication.base']}"
      data-sly-use.clientLib="/libs/granite/sightly/templates/clientlib.html"/>
   <coral-shell-header aria-hidden="false" aria-label="Header Bar"
      class="coral--dark granite-shell-header coral3-Shell-header"
      role="region">
      <coral-shell-header-home aria-level="2" class="globalnav-toggle" data-globalnav-toggle-href="/" role="heading">
         <a class="coral3-Shell-homeAnchor" href="/"
            icon="adobeExperienceManagerColor" is="coral-shell-homeanchor"
            style="display: inline-block; padding-right: 0;">
            <coral-icon
               aria-label="adobe experience manager color"
               class="coral3-Icon coral3-Shell-homeAnchor-icon coral3-Icon--adobeExperienceManagerColor coral3-Icon--sizeM"
               icon="adobeExperienceManagerColor" role="img"
               size="M"></coral-icon>
            <coral-shell-homeanchor-label>Adobe Experience Manager
            </coral-shell-homeanchor-label>
         </a>
         <span style="line-height: 2.375rem;">/ AEM Operations / ${properties.jcr:title} </span>
      </coral-shell-header-home>
   </coral-shell-header>
   <div class="foundation-layout-panel">
      <div class="foundation-layout-panel-header">
         <betty-titlebar>
            <betty-titlebar-title>
               <span aria-level="1" class="granite-title" role="heading">${properties.jcr:title}</span>
            </betty-titlebar-title>
            <betty-titlebar-primary></betty-titlebar-primary>
            <betty-titlebar-secondary>
            </betty-titlebar-secondary>
         </betty-titlebar>
      </div>
      <div class="content-container-inner coral-Well" style="width:60%; margin:0 auto;">
         <div class="acsCommons-System-Notifications-Form-row">
		   <label class="coral-Form-fieldlabel">List of Paths</label>
		   <label> Note: Add the list of paths has a Line feed (No need for comma separation)</label>
		   <textarea
		      class="coral-Textfield coral-Textfield--multiline acsCommons-System-Notifications-Page-input--textarea"
		      rows="6"
		      name="./pathsList"></textarea>
		 </div>
		 <label> Note: Create a Excel Sheet with column called <b>destination</b> and the list of values and upload</label>
         <sly data-sly-use.packprops="com.aem.operations.core.models.ListTreeActivationModel">
            <sly data-sly-resource="${packprops.formResource}"/>
         </sly>
         <coral-actionbar-item class="coral3-ActionBar-item">
            <button class="coral3-Button coral3-Button--primary activation-initiator" icon="checkCircle" iconsize="S"
               is="coral-button"
               size="M" variant="primary">
               <coral-icon alt="" class="coral3-Icon coral3-Icon--sizeS coral3-Icon--checkCircle" icon="publish"
                  size="S"></coral-icon>
               <coral-button-label>
                  Start the Replication
               </coral-button-label>
            </button>
         </coral-actionbar-item>
          <!-- The Modal -->
          <div id="popup-modal" class="modal">
              <!-- Modal content -->
              <div class="modal-content">
                  <span class="close">&times;</span>
                  <span class="loading">Loading</span>
                  <span class="loader"></span>
              </div>
          </div>
      </div>
   </div>
</body>

Add the following JS code to process the ajax request as shown below:

var Coral = window.Coral || {},
    Granite = window.Granite || {};

(function (window, document, $, Coral) {
    "use strict";
    $(document).on("foundation-contentloaded", function (e) {

        var SITE_PATH = "/conf/aemoperations/settings/tools/listreplication-initiator.html",
            ui = $(window).adaptTo("foundation-ui");

        if (window.location.href.indexOf(SITE_PATH) < 0) {
            return;
        }

        $(document).off("change", ".close").on("change", ".close", function (event) {
            $(".modal").hide();
        });

        function getValueByName(fieldName, isMandatory) {
            var fieldValue = ($("input[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("input[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("input[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getValueByNameArea(fieldName, isMandatory) {
            var fieldValue = ($("textarea[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("textarea[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("textarea[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getFileByName(fieldName) {
            var fieldInput = $("input[name='" + fieldName + "']");
            if(null != fieldInput && null != fieldInput[0]){
                return fieldInput[0].files[0];
            }
        }

        $(document).off("click", ".activation-initiator").on("click", ".activation-initiator", function (event) {
            event.preventDefault();

            var queueMethod = getValueByName('./queueMethod', false),
                agents = getValueByName('./agents', false),
                reAction = getValueByName('./reAction', false),
                pathsList = getValueByNameArea('./pathsList', true),
                repPathExcel = getFileByName('./repPathExcel');

            var formData = new FormData();
            if(null != repPathExcel){
                formData.append("file", repPathExcel, repPathExcel.name);
            }
            formData.append("queueMethod", queueMethod);
            formData.append("agents", agents);
            formData.append("reAction", reAction);
            formData.append("pathsList", pathsList);

			$(".loading").html("Replication in progress");
            $(".modal").show();

            $.ajax({
                url: "/bin/triggerListActivation",
                method: "POST",
                async: true,
                cache: false,
                contentType: false,
                processData: false,
                data: formData
            }).done(function (data) {
                if (data && data.message){
                    ui.notify("Success", data.message, "success");
                    var dialog = new Coral.Dialog();
                    dialog.id = 'dialogSuccess';
                    dialog.header.innerHTML = 'Success';
                    dialog.content.innerHTML = data.message;
                    dialog.footer.innerHTML = '<button class="ok-button" is="coral-button" variant="primary" icon="check" coral-close>OK</button>';
                    dialog.variant = 'success';
                    dialog.closable = "on";
                    dialog.show();
                    $(".modal").hide();
                }else{
                    ui.notify("Error", "Unable to process List Activation", "error");
                }
            }).fail(function (data) {
				$(".modal").hide();
                if (data && data.responseJSON && data.responseJSON.message){
                    ui.notify("Error", data.responseJSON.message, "error");
                }else{
                    ui.notify("Error", "Unable to process List Activation", "error");
                }
            });
        });
    });
})(window, document, $, Coral);

Once the code is built and deployed you will be able to access the tool by navigating the:

Tools -> {Tool Section} -> {Tool Name}

List Process Section

Now you can input the page paths as a line feed (no need to separate the paths using commas or any characters)

List Process Page

Upload the excel sheet or paths and select the queue method it’s recommended to use MCP Queue if you are activating more than 10K pages then it’s recommended to select MCP After 10K.

Select the Action to be performed.

List Replication Result
Page activation result

Working Code:
You can also access the working code on my GitHub repository link: https://github.com/kiransg89/aemoperations

Note: You can access this package over domain on AEM / AEMaaCS and over any environments and architects can handle the permissions by adding appropriate rep:policy on cq generator node or conf page

AEM Tool – Perfect Content backup Package in AEM

Problem statement:

Create a perfect content backup package in AEM, whenever we want to create any package in AEM we provide the content paths in the package filter manually and what do we get?

  1. We get only content pages.
  2. What about images and referenced pages?
  3. What about header/footer experience fragment pages or XF page-related context hub variations?

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

  1. AEM operations
  2. Managing templates
  3. Cloud configurations
  4. ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Perfect packager pulls the current page resource and iterates on each node and gets the referenced images, pages, and XF’s. Once all the paths are pulled it reiterates to find all the referenced pages to get all the images, XF’s, and pages.

In order to create a perfect content backup package please use the Tool generator to give your tool name and descriptions: https://kiransg.com/2022/11/24/aem-tool-create-generate-tool-from-scratch

Add the following Sling model fields as shown below to accept the inputs:

package com.aem.operations.core.models;

import javax.inject.Named;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.GeneratedDialog;
import lombok.Getter;
import lombok.Setter;

@Model(adaptables = { Resource.class,SlingHttpServletRequest.class }, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class PerfectPackageModel extends GeneratedDialog {

    @Getter
    @Setter
    @Named(value = "packageName")
    @FormField(
            name = "Package Name",
            category = "General",
            required = true,
            description = "Enter the package name",
            hint = "Name")
    private String packageName;

    @Getter
    @Setter
    @Named(value = "packageDescription")
    @FormField(
            name = "Package Description",
            category = "General",
            required = true,
            description = "Enter the package Description",
            hint = "Description")
    private String packageDescription;
}

Add the following Servlet to process the request as shown below:

package com.aem.operations.core.servlets;

import java.io.IOException;
import java.io.PrintWriter;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import com.aem.operations.core.utils.ReportRetryUtils;
import com.aem.operations.core.visitors.ContentVisitor;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.packaging.JcrPackage;
import org.apache.jackrabbit.vault.packaging.JcrPackageDefinition;
import org.apache.jackrabbit.vault.packaging.JcrPackageManager;
import org.apache.jackrabbit.vault.packaging.PackageException;
import org.apache.jackrabbit.vault.packaging.Packaging;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.sling.servlets.post.JSONResponse;
import org.jetbrains.annotations.NotNull;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.acs.commons.packaging.PackageHelper;
import com.adobe.granite.rest.Constants;
import com.google.gson.Gson;
import com.google.gson.JsonObject;

@Component(
        service = { Servlet.class },
        property = {
                "sling.servlet.paths=" + PerfectPackageServlet.RESOURCE_PATH,
                "sling.servlet.methods=POST"
        }
)
public class PerfectPackageServlet extends SlingAllMethodsServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger LOGGER = LoggerFactory.getLogger(PerfectPackageServlet.class);
    public static final String RESOURCE_PATH = "/bin/triggerPerfectPackage";
    private static final String MESSAGE = "message";

    private static final String DEFUALT_GROUP_NAME = "my_packages";
    private static final String DEFUALT_VERSION = "1.0";
    private static final String QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH = "/apps/acs-commons/components/utilities/packager/query-packager/definition/package-thumbnail.png";

    @Reference
    private PackageHelper packageHelper;

    @Reference
    private Packaging packaging;

    @Override
    protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response) throws ServletException, IOException {
        // Set response headers.
        response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE);
        response.setCharacterEncoding(Constants.DEFAULT_CHARSET);

        JsonObject jsonResponse = new JsonObject();

        String packageName = request.getParameter("packageName");
        String packageDescription = request.getParameter("packageGroup");
        String pathsList = request.getParameter("pathsList");

        String packagePath = StringUtils.EMPTY;
        if(StringUtils.isNotEmpty(pathsList)) {
            packagePath = collectReferences(request.getResourceResolver(), pathsList, packageName, packageDescription);
        }
        if(StringUtils.isNotEmpty(packagePath)) {
            jsonResponse.addProperty(MESSAGE, "Package creation is Successful and path is <a href=\"" + packagePath + "\">" + packagePath + "</a>");
        } else {
            LOGGER.error("Unable to process package creation");
        }
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }

    private String collectReferences(ResourceResolver resourceResolver, String pagePaths, String packageName, String packageDescription) {
        Set<String> contentPaths = new HashSet<>();
        Set<String> xfPaths = new HashSet<>();
        Set<String> damPaths = new HashSet<>();
        ContentVisitor contentVisitor =  new ContentVisitor();
        Set<String> pages = new HashSet<>(Arrays.asList(StringUtils.split(pagePaths, '\n'))).stream().map(r -> r.trim() + "/jcr:content").collect(Collectors.toSet());
        try {
            iterateContentPaths(resourceResolver, contentPaths, xfPaths, damPaths, contentVisitor, pages);
            damPaths.addAll(iterateContent(resourceResolver, contentVisitor, damPaths));
            ReportRetryUtils.withRetry(2, 300, () -> xfPaths.addAll(iterateContent(resourceResolver, contentVisitor, xfPaths)));
            Set<String> allPaths = Stream.of(pages, contentPaths, xfPaths, damPaths).flatMap(Collection::stream).collect(Collectors.toSet());
            return packagePaths(resourceResolver, allPaths, packageName, packageDescription);
        } catch (Exception e) {
            LOGGER.error("Exception occurred during package creation in Retry {}", e.getMessage());
        }
        return StringUtils.EMPTY;
    }

    private Set<String> iterateContent(ResourceResolver resourceResolver, ContentVisitor contentVisitor, Set<String> pages) {
        Set<String> interimPaths = new HashSet<>();
        for(String page : pages) {
            contentVisitor.accept(resourceResolver.resolve(page));
            interimPaths.addAll(contentVisitor.getContentPaths());
            interimPaths.addAll(contentVisitor.getDamPaths());
            interimPaths.addAll(contentVisitor.getXfPaths());
        }
        return interimPaths;
    }

    private void iterateContentPaths(ResourceResolver resourceResolver, Set<String> contentPaths, Set<String> xfPaths,
                                     Set<String> damPaths, ContentVisitor contentVisitor, Set<String> pages) {
        for(String page : pages) {
            contentVisitor.accept(resourceResolver.resolve(page));
            contentPaths.addAll(contentVisitor.getContentPaths());
            xfPaths.addAll(contentVisitor.getXfPaths());
            damPaths.addAll(contentVisitor.getDamPaths());
        }
    }

    private String packagePaths(ResourceResolver resourceResolver, Set<String> allPaths, String packageName, String packageDescription)
            throws IOException, RepositoryException, PackageException {
        Map<String, String> packageDefinitionProperties = new HashMap<>();
        // ACL Handling
        packageDefinitionProperties.put(JcrPackageDefinition.PN_AC_HANDLING,
                AccessControlHandling.OVERWRITE.toString());

        // Package Description
        packageDefinitionProperties.put(JcrPackageDefinition.PN_DESCRIPTION, packageDescription);
        Set<@NotNull Resource> packageResources = allPaths.stream().map(resourceResolver::resolve).collect(Collectors.toSet());
        try(JcrPackage jcrPackage = packageHelper.createPackage(packageResources,
                resourceResolver.adaptTo(Session.class), DEFUALT_GROUP_NAME, packageName, DEFUALT_VERSION,
                PackageHelper.ConflictResolution.Replace, packageDefinitionProperties)){
            // Add thumbnail to the package definition
            packageHelper.addThumbnail(jcrPackage, resourceResolver.getResource(QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH));
            final JcrPackageManager packageManager = packaging.getPackageManager(resourceResolver.adaptTo(Session.class));
            packageManager.assemble(jcrPackage, null);
            return jcrPackage.getNode().getPath();
        }
    }
}

Add the following Content Visitor class to iterate resources as shown below:

package com.aem.operations.core.visitors;

import java.util.Arrays;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
import java.util.regex.Pattern;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.resource.AbstractResourceVisitor;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceUtil;
import org.apache.sling.api.resource.ValueMap;
import org.jetbrains.annotations.NotNull;

public class ContentVisitor extends AbstractResourceVisitor {

    private static final String CONTENT_SLASH = "/content/";
    private static final String CONTENT_DAM_SLASH = "/content/dam/";
    private static final String CONTENT_XF_SLASH = "/content/experience-fragments/";

    Pattern htmlPattern = Pattern.compile(".*\\<[^>]+>.*", Pattern.DOTALL);

    Set<String> contentPaths = new HashSet<>();
    Set<String> xfPaths = new HashSet<>();
    Set<String> damPaths = new HashSet<>();

    @Override
    public final void accept(final Resource resource) {
        if (null != resource && !ResourceUtil.isNonExistingResource(resource)) {
            final ValueMap properties = resource.adaptTo(ValueMap.class);
            final String primaryType = properties.get(ResourceResolver.PROPERTY_RESOURCE_TYPE, StringUtils.EMPTY);
            if(StringUtils.isNoneEmpty(primaryType) || StringUtils.startsWith(resource.getPath(), "/content/dam/content-fragments")){
                visit(resource);
            }
            this.traverseChildren(resource.listChildren());
        }
    }

    @Override
    protected void traverseChildren(final @NotNull Iterator<Resource> children) {
        while (children.hasNext()) {
            final Resource child = children.next();
            accept(child);
        }
    }

    @Override
    protected void visit(final @NotNull Resource resource) {
        resource.getValueMap().entrySet().forEach(property -> {
            Object prop = property.getValue();
            if (prop.getClass() == String[].class) {
                List<String> propertyValue = Arrays.asList((String[]) prop);
                if (!propertyValue.isEmpty()) {
                    propertyValue.stream().filter(s -> StringUtils.isNotEmpty(s) && StringUtils.startsWith(s, CONTENT_SLASH) && !htmlPattern.matcher(s).matches()).forEach(this::populateValue);
                }
            } else if (prop.getClass() == String.class) {
                String propertyValue = (String) prop;
                if (StringUtils.isNotEmpty(propertyValue) && StringUtils.startsWith(propertyValue, CONTENT_SLASH) && !htmlPattern.matcher(propertyValue).matches()) {
                    populateValue(propertyValue);
                }
            }
        });
    }

    private void populateValue(String value) {
        if(StringUtils.startsWith(value, CONTENT_DAM_SLASH)) {
            damPaths.add(value);
        } else if(StringUtils.startsWith(value, CONTENT_XF_SLASH)) {
            xfPaths.add(value+"/jcr:content");
        } else {
            contentPaths.add(StringUtils.substringBeforeLast(value, ".html")+"/jcr:content");
        }
    }

    public Set<String> getContentPaths() {
        return contentPaths;
    }

    public Set<String> getXfPaths() {
        return xfPaths;
    }

    public Set<String> getDamPaths() {
        return damPaths;
    }
}

Add the following Report Retry Utils as shown below:

package com.aem.operations.core.utils;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class ReportRetryUtils {
    private static final Logger log = LoggerFactory.getLogger(ReportRetryUtils.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int i = 0; i < maxTimes; i++) {
            try {
                call.process();
                if(i == (maxTimes-1)) {
                    return true;
                }
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (i + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

Add the following HTL to render the page:

<!DOCTYPE html>
<head>
   <link rel="shortcut icon" href="/libs/granite/core/content/login/favicon.ico">
</head>
<body class="coral--light foundation-layout-util-maximized-alt">
   <sly data-sly-call="${clientLib.all @ categories=['coralui3','granite.ui.coral.foundation','granite.ui.shell','cq.authoring.dialog', 'perfectpackage.base']}"
      data-sly-use.clientLib="/libs/granite/sightly/templates/clientlib.html"/>
   <coral-shell-header aria-hidden="false" aria-label="Header Bar"
      class="coral--dark granite-shell-header coral3-Shell-header"
      role="region">
      <coral-shell-header-home aria-level="2" class="globalnav-toggle" data-globalnav-toggle-href="/" role="heading">
         <a class="coral3-Shell-homeAnchor" href="/"
            icon="adobeExperienceManagerColor" is="coral-shell-homeanchor"
            style="display: inline-block; padding-right: 0;">
            <coral-icon
               aria-label="adobe experience manager color"
               class="coral3-Icon coral3-Shell-homeAnchor-icon coral3-Icon--adobeExperienceManagerColor coral3-Icon--sizeM"
               icon="adobeExperienceManagerColor" role="img"
               size="M"></coral-icon>
            <coral-shell-homeanchor-label>Adobe Experience Manager
            </coral-shell-homeanchor-label>
         </a>
         <span style="line-height: 2.375rem;">/ AEM Operations / ${properties.jcr:title} </span>
      </coral-shell-header-home>
   </coral-shell-header>
   <div class="foundation-layout-panel">
      <div class="foundation-layout-panel-header">
         <betty-titlebar>
            <betty-titlebar-title>
               <span aria-level="1" class="granite-title" role="heading">${properties.jcr:title}</span>
            </betty-titlebar-title>
            <betty-titlebar-primary></betty-titlebar-primary>
            <betty-titlebar-secondary>
            </betty-titlebar-secondary>
         </betty-titlebar>
      </div>
      <div class="content-container-inner coral-Well" style="width:60%; margin:0 auto;">
         <div class="acsCommons-System-Notifications-Form-row">
		   <label class="coral-Form-fieldlabel">List of Paths</label>
		   <label> Note: Add the list of paths has a Line feed (No need for comma separation)</label>
		   <textarea
		      class="coral-Textfield coral-Textfield--multiline acsCommons-System-Notifications-Page-input--textarea"
		      rows="6"
		      name="./pathsList"></textarea>
		</div>
         <sly data-sly-use.packprops="com.aem.operations.core.models.PerfectPackageModel">
            <sly data-sly-resource="${packprops.formResource}"/>
         </sly>
         <coral-actionbar-item class="coral3-ActionBar-item">
            <button class="coral3-Button coral3-Button--primary package-initiator" icon="checkCircle" iconsize="S"
               is="coral-button"
               size="M" variant="primary">
               <coral-icon alt="" class="coral3-Icon coral3-Icon--sizeS coral3-Icon--checkCircle" icon="folderGear"
                  size="S"></coral-icon>
               <coral-button-label>
                  Create Content Backup
               </coral-button-label>
            </button>
         </coral-actionbar-item>
          <!-- The Modal -->
          <div id="popup-modal" class="modal">
              <!-- Modal content -->
              <div class="modal-content">
                  <span class="close">&times;</span>
                  <span class="loading">Loading</span>
                  <span class="loader"></span>
              </div>
          </div>
      </div>
   </div>
</body>

Add the following JS code to process the ajax request as shown below:

var Coral = window.Coral || {},
    Granite = window.Granite || {};

(function (window, document, $, Coral) {
    "use strict";
    $(document).on("foundation-contentloaded", function (e) {

        var SITE_PATH = "/conf/aemoperations/settings/tools/perfectpackage-initiator.html",
            ui = $(window).adaptTo("foundation-ui");

        if (window.location.href.indexOf(SITE_PATH) < 0) {
            return;
        }

        $(document).off("change", ".close").on("change", ".close", function (event) {
            $(".modal").hide();
        });

        function getValueByName(fieldName, isMandatory) {
            var fieldValue = ($("input[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("input[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("input[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getValueByNameArea(fieldName, isMandatory) {
            var fieldValue = ($("textarea[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("textarea[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("textarea[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        $(document).off("click", ".package-initiator").on("click", ".package-initiator", function (event) {
            event.preventDefault();

            var packageName = getValueByName('./packageName', true),
                packageDescription = getValueByName('./packageDescription', true),
                pathsList = getValueByNameArea('./pathsList', true);

            var formData = new FormData();
            formData.append("packageName", packageName);
            formData.append("packageDescription", packageDescription);
            formData.append("pathsList", pathsList);

			$(".loading").html("PLEASE WAIT CREATINGING PACKAGE");
            $(".modal").show();

            $.ajax({
                url: "/bin/triggerPerfectPackage",
                method: "POST",
                async: true,
                cache: false,
                contentType: false,
                processData: false,
                data: formData
            }).done(function (data) {
                if (data && data.message){
                    ui.notify("Success", data.message, "success");
                    var dialog = new Coral.Dialog();
                    dialog.id = 'dialogSuccess';
                    dialog.header.innerHTML = 'Success';
                    dialog.content.innerHTML = data.message;
                    dialog.footer.innerHTML = '<button class="ok-button" is="coral-button" variant="primary" icon="check" coral-close>OK</button>';
                    dialog.variant = 'success';
                    dialog.closable = "on";
                    dialog.show();
                    $(".modal").hide();
                }else{
                    ui.notify("Error", "Unable to create package", "error");
                }
            }).fail(function (data) {
				$(".modal").hide();
                if (data && data.responseJSON && data.responseJSON.message){
                    ui.notify("Error", data.responseJSON.message, "error");
                }else{
                    ui.notify("Error", "Unable to create package", "error");
                }
            });
        });
    });
})(window, document, $, Coral);

Once the code is built and deployed you will be able to access the tool by navigating the:

Tools -> {Tool Section} -> {Tool Name}

Perfect Package section

Now you can input the page paths as a line feed (no need to separate the paths using commas or any characters)

Perfect Package Result

Provide the desired package name and description and you can see the package path will be prompted and you will be able to download the built package by clicking on the link:

Perfect Package result
Perfect package built and has all the references

Working Code:
You can also access the working code on my GitHub repository link: https://github.com/kiransg89/aemoperations

Note: You can access this package over domain on AEM / AEMaaCS and over any environments and architects can handle the permissions by adding appropriate rep:policy on cq generator node or conf page

AEM Tool – Create a package using a list of paths

Problem statement:

How to create a package using a list of paths?

Avoid adding individual paths manually in the CRX Package manager

Option to package current page or pull children

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

  1. AEM operations
  2. Managing templates
  3. Cloud configurations
  4. ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Package Creator is an extension of the ACS Commons Package handler, but it’s UI friendly and easy to access. You can input the paths to be packaged as a line feed and no need to separate them by a comma. Once the package is ready it will be built and prompted with a download option.

In order to create a package please use the Tool generator to give your tool name and descriptions: https://kiransg.com/2022/11/24/aem-tool-create-generate-tool-from-scratch

Add the following Sling model fields as shown below to accept the inputs:

package com.aem.operations.core.models;

import javax.inject.Named;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.GeneratedDialog;
import com.adobe.acs.commons.mcp.form.SelectComponent;
import lombok.Getter;
import lombok.Setter;

@Model(adaptables = { Resource.class,SlingHttpServletRequest.class }, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class PackageCreatorModel extends GeneratedDialog {

    public enum GetChildren {
        NO, YES
    }

    @Getter
    @Setter
    @Named(value = "packageName")
    @FormField(
            name = "Package Name",
            category = "General",
            required = true,
            description = "Enter the package name",
            hint = "Name")
    private String packageName;

    @Getter
    @Setter
    @Named(value = "packageDescription")
    @FormField(
            name = "Package Description",
            category = "General",
            required = true,
            description = "Enter the package Description",
            hint = "Description")
    private String packageDescription;

    @Getter
    @Setter
    @Named(value = "getChildren")
    @FormField(
            name = "Get Children",
            description = "child pages will be pulled",
            required = true,
            component = SelectComponent.EnumerationSelector.class,
            options = {"horizontal", "default=NO"})
    GetChildren getChildren = GetChildren.NO;
}

Add the following Servlet to process the request as shown below:

package com.aem.operations.core.servlets;

import com.adobe.acs.commons.packaging.PackageHelper;
import com.adobe.granite.rest.Constants;
import com.aem.operations.core.models.PackageCreatorModel;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.packaging.*;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.sling.servlets.post.JSONResponse;
import org.jetbrains.annotations.Nullable;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.*;
import java.util.stream.Collectors;

@Component(
        service = { Servlet.class },
        property = {
                "sling.servlet.paths=" + PackageCreatorServlet.RESOURCE_PATH,
                "sling.servlet.methods=POST"
        }
)
public class PackageCreatorServlet extends SlingAllMethodsServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger LOGGER = LoggerFactory.getLogger(PackageCreatorServlet.class);
    public static final String RESOURCE_PATH = "/bin/triggerPackageCreator";
    private static final String MESSAGE = "message";
    private static final String CONTENT_DAM_SLASH = "/content/dam/";

    private static final String DEFUALT_GROUP_NAME = "my_packages";
    private static final String DEFUALT_VERSION = "1.0";
    private static final String QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH = "/apps/acs-commons/components/utilities/packager/query-packager/definition/package-thumbnail.png";

    @Reference
    private PackageHelper packageHelper;

    @Reference
    private Packaging packaging;

    @Override
    protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response) throws ServletException, IOException {
        // Set response headers.
        response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE);
        response.setCharacterEncoding(Constants.DEFAULT_CHARSET);

        JsonObject jsonResponse = new JsonObject();
        ResourceResolver resourceResolver = request.getResourceResolver();

        String packageName = request.getParameter("packageName");
        String packageDescription = request.getParameter("packageGroup");
        String pathsList = request.getParameter("pathsList");
        String getChildren = request.getParameter("getChildren");

        String packagePath = StringUtils.EMPTY;
        if(StringUtils.isNotEmpty(pathsList)) {
            packagePath = createPackage(resourceResolver, packageName, packageDescription, pathsList, getChildren);
        }

        if(StringUtils.isNotEmpty(packagePath)) {
            jsonResponse.addProperty(MESSAGE, "Package creation is Successful and path is <a href=\"" + packagePath + "\">" + packagePath + "</a>");
        } else {
            LOGGER.error("Unable to process package creation");
        }
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }

    private String createPackage(ResourceResolver resourceResolver, String packageName, String packageDescription,
                                 String pathsList, String getChildren) throws IOException {
        Set<String> pages = new HashSet<>(Arrays.asList(StringUtils.split(pathsList, '\n'))).stream().map(String::trim).collect(Collectors.toSet());
        Set<Resource> packageResources = pages.stream().map(path -> getResource(resourceResolver, path, getChildren)).collect(Collectors.toSet());
        if(null != packageResources && !packageResources.isEmpty()) {
            Map<String, String> packageDefinitionProperties = new HashMap<>();
            // ACL Handling
            packageDefinitionProperties.put(JcrPackageDefinition.PN_AC_HANDLING,
                    AccessControlHandling.OVERWRITE.toString());
            // Package Description
            packageDefinitionProperties.put(JcrPackageDefinition.PN_DESCRIPTION, packageDescription);
            try(JcrPackage jcrPackage = packageHelper.createPackage(packageResources,
                    resourceResolver.adaptTo(Session.class), DEFUALT_GROUP_NAME, packageName, DEFUALT_VERSION,
                    PackageHelper.ConflictResolution.Replace, packageDefinitionProperties)){
                // Add thumbnail to the package definition
                packageHelper.addThumbnail(jcrPackage, resourceResolver.getResource(QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH));
                final JcrPackageManager packageManager = packaging.getPackageManager(resourceResolver.adaptTo(Session.class));
                packageManager.assemble(jcrPackage, null);
                return jcrPackage.getNode().getPath();
            } catch (RepositoryException | PackageException e) {
                LOGGER.error("Exception occurred during package creation {}", e.getMessage());
            }
        }
        return StringUtils.EMPTY;
    }

    private @Nullable Resource getResource(ResourceResolver resourceResolver, String path, String getChildren) {
        if(StringUtils.equalsIgnoreCase(PackageCreatorModel.GetChildren.NO.toString(), getChildren) && !StringUtils.contains(path, CONTENT_DAM_SLASH)) {
            return resourceResolver.resolve(path + "/jcr:content");
        }
        return resourceResolver.resolve(path);
    }
}

Add the following HTL to render the page:

<!DOCTYPE html>
<head>
   <link rel="shortcut icon" href="/libs/granite/core/content/login/favicon.ico">
</head>
<body class="coral--light foundation-layout-util-maximized-alt">
   <sly data-sly-call="${clientLib.all @ categories=['coralui3','granite.ui.coral.foundation','granite.ui.shell','cq.authoring.dialog', 'packagecreator.base']}"
      data-sly-use.clientLib="/libs/granite/sightly/templates/clientlib.html"/>
   <coral-shell-header aria-hidden="false" aria-label="Header Bar"
      class="coral--dark granite-shell-header coral3-Shell-header"
      role="region">
      <coral-shell-header-home aria-level="2" class="globalnav-toggle" data-globalnav-toggle-href="/" role="heading">
         <a class="coral3-Shell-homeAnchor" href="/"
            icon="adobeExperienceManagerColor" is="coral-shell-homeanchor"
            style="display: inline-block; padding-right: 0;">
            <coral-icon
               aria-label="adobe experience manager color"
               class="coral3-Icon coral3-Shell-homeAnchor-icon coral3-Icon--adobeExperienceManagerColor coral3-Icon--sizeM"
               icon="adobeExperienceManagerColor" role="img"
               size="M"></coral-icon>
            <coral-shell-homeanchor-label>Adobe Experience Manager
            </coral-shell-homeanchor-label>
         </a>
         <span style="line-height: 2.375rem;">/ AEM Operations / ${properties.jcr:title} </span>
      </coral-shell-header-home>
   </coral-shell-header>
   <div class="foundation-layout-panel">
      <div class="foundation-layout-panel-header">
         <betty-titlebar>
            <betty-titlebar-title>
               <span aria-level="1" class="granite-title" role="heading">${properties.jcr:title}</span>
            </betty-titlebar-title>
            <betty-titlebar-primary></betty-titlebar-primary>
            <betty-titlebar-secondary>
            </betty-titlebar-secondary>
         </betty-titlebar>
      </div>
      <div class="content-container-inner coral-Well" style="width:60%; margin:0 auto;">
         <div class="acsCommons-System-Notifications-Form-row">
		   <label class="coral-Form-fieldlabel">List of Paths</label>
		   <label> Note: Add the list of paths has a Line feed (No need for comma separation)</label>
		   <textarea
		      class="coral-Textfield coral-Textfield--multiline acsCommons-System-Notifications-Page-input--textarea"
		      rows="6"
		      name="./pathsList"></textarea>
		 </div>
         <sly data-sly-use.packprops="com.aem.operations.core.models.PackageCreatorModel">
            <sly data-sly-resource="${packprops.formResource}"/>
         </sly>
         <coral-actionbar-item class="coral3-ActionBar-item">
            <button class="coral3-Button coral3-Button--primary package-initiator" icon="checkCircle" iconsize="S"
               is="coral-button"
               size="M" variant="primary">
               <coral-icon alt="" class="coral3-Icon coral3-Icon--sizeS coral3-Icon--checkCircle" icon="actions"
                  size="S"></coral-icon>
               <coral-button-label>
                  Create Package
               </coral-button-label>
            </button>
         </coral-actionbar-item>
          <!-- The Modal -->
          <div id="popup-modal" class="modal">
              <!-- Modal content -->
              <div class="modal-content">
                  <span class="close">&times;</span>
                  <span class="loading">Loading</span>
                  <span class="loader"></span>
              </div>
          </div>
      </div>
   </div>
</body>

Add the following JS code to process the ajax request as shown below:

var Coral = window.Coral || {},
    Granite = window.Granite || {};

(function (window, document, $, Coral) {
    "use strict";
    $(document).on("foundation-contentloaded", function (e) {

        var SITE_PATH = "/conf/aemoperations/settings/tools/create-package-initiator.html",
            ui = $(window).adaptTo("foundation-ui");

        if (window.location.href.indexOf(SITE_PATH) < 0) {
            return;
        }

        $(document).off("change", ".close").on("change", ".close", function (event) {
            $(".modal").hide();
        });

        function getValueByName(fieldName, isMandatory) {
            var fieldValue = ($("input[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("input[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("input[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getValueByNameArea(fieldName, isMandatory) {
            var fieldValue = ($("textarea[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("textarea[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("textarea[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        $(document).off("click", ".package-initiator").on("click", ".package-initiator", function (event) {
            event.preventDefault();

            var packageName = getValueByName('./packageName', true),
                packageDescription = getValueByName('./packageDescription', true),
                pathsList = getValueByNameArea('./pathsList', true),
                getChildren = getValueByName('./getChildren', true);

            var formData = new FormData();
            formData.append("packageName", packageName);
            formData.append("packageDescription", packageDescription);
            formData.append("pathsList", pathsList);
            formData.append("getChildren", getChildren);

			$(".loading").html("PLEASE WAIT CREATINGING PACKAGE");
            $(".modal").show();

            $.ajax({
                url: "/bin/triggerPackageCreator",
                method: "POST",
                async: true,
                cache: false,
                contentType: false,
                processData: false,
                data: formData
            }).done(function (data) {
                if (data && data.message){
                    ui.notify("Success", data.message, "success");
                    var dialog = new Coral.Dialog();
                    dialog.id = 'dialogSuccess';
                    dialog.header.innerHTML = 'Success';
                    dialog.content.innerHTML = data.message;
                    dialog.footer.innerHTML = '<button class="ok-button" is="coral-button" variant="primary" icon="check" coral-close>OK</button>';
                    dialog.variant = 'success';
                    dialog.closable = "on";
                    dialog.show();
                    $(".modal").hide();
                }else{
                    ui.notify("Error", "Unable to create package ", "error");
                }
            }).fail(function (data) {
				$(".modal").hide();
                if (data && data.responseJSON && data.responseJSON.message){
                    ui.notify("Error", data.responseJSON.message, "error");
                }else{
                    ui.notify("Error", "Unable to create package", "error");
                }
            });
        });
    });
})(window, document, $, Coral);

Once the code is built and deployed you will be able to access the tool by navigating the:

Tools -> {Tool Section} -> {Tool Name}

Package Creator

Now you can input the page paths as a line feed (no need to separate the paths using commas or any characters)

Creator Page

Provide the list paths to be backed up, desired package name, and description and select the dropdown to pull children or not. Once the package is built you can see the package path will be prompted and you will be able to download the package by clicking on the built package link:

Package Creator Results
The package is built with the exact page location

Working Code:
You can also access the working code on my GitHub repository link: https://github.com/kiransg89/aemoperations

Note: You can access this package over domain on AEM / AEMaaCS and over any environments and architects can handle the permissions by adding appropriate rep:policy on cq generator node or conf page

AEM Tool – Create / Generate tool from scratch

Problem Statement:

Create a tool similar to AEM Operations or ACS Commons for easy access and to run any maintenance process/tasks.

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

  1. AEM operations
  2. Managing templates
  3. Cloud configurations
  4. ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

In order to create any tool from scratch takes a lot of time and man hours and once all the configs are ready then it takes more time to develop the services and servlet to handle the business logic.

By using this tool, you can avoid all kinds of configurations and initial setup and kick start with your own first tool from scratch

The tool generator consists of the:

  1. Sling model – to generate fields and handle inputs
  2. Servlet – to process request
  3. Component – to handle view (HTL), CSS, and JS

All the above sample scratch setups along with ready to check-in code will be added to the code base.

How to use the tool?

1. Click on the link to download the tool generator package and install it into your local instance
2. Once the package is installed go to sites -> tools -> and select the Tool Generator section
tool section
3. Select Tool Settings on the top right hand corner and provide your local repository paths like sling model path, servlet path, apps path, conf path, and CQ path if it already exists
tool page
4. Once all the settings are authored save the settings
tool settings
5. After coming to the generator page provide your tool name and tool description
tool authoring
6. If you have an existing tool then select Yes else Select No and provide the Tool Section name
7. Click on Create tool to create your new tool from scratch

Check your repository for all the file changes as shown below:

You can also check your new tool component and other configs on CRXDE

You can also visit your new tool by accessing the tool section (sites -> Tools -> {Your section name})

For more info on how to add a Sling model field please visit the below link:

Once your Sling model, servlet, and other things are ready make sure you add the following filter in your META-INF folder and keep it merge as shown below:

This would avoid replacing/overriding any other tools like ACS Commons or your other repo tools.