Delete Unreferenced Assets AEM

Problem Statement:

Delete all the assets which don’t have references to improve AEM performance in turn Indexes and search/query performance.

Introduction:

How do assets get published?

The author uploads the images and publishes the assets
Create a launcher and workflow which processes assets metadata and publishes the pages
Whenever we publish any pages and if the page has references to assets, then during publishing, it asks to replicate the references as well.

What happens when the page is unpublished?

When the page is deactivated, assets referenced to the page will not be deactivated because this asset might have reference to the other pages hence out of the box assets won’t be deactivated.
If we perform cleanup, deactivate and delete old pages, we might not be cleaning up assets related to this page.

What advantages of cleaning up of old assets?

Drastically reduces repository size
Improves DAM Asset search
Improves indexing

Generate Published Asset Report by visiting:

Go to Tools -> Assets -> Reports as shown below:

Click on create and click on Publish report

Provide folder path and start date and end date

Select the columns as per the requirement

Finally, the report will be ready with all the assets lists as shown below

Download the report to see the final list of images

To generate the report let’s create a new AEM tool:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

AEM operations
Managing templates
Cloud configurations
ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Broken Asset Report generates a report of all the unreferenced assets by running a Reference Search query across the repository every 30s (by default), you can update the scheduler expression based on your repository size.

The scheduler also checks for the CPU/HEAP size before triggering the reference search process and for more details on Throttled scheduler please refer to the link.

In order to create a perfect content backup package please use the Tool generator to give your tool name and descriptions: https://kiransg.com/2022/11/24/aem-tool-create-generate-tool-from-scratch

Please refer to my GitHub repository for working code on Broken Asset Reference:

Once the repository is built and deployed, you will be able to access the Broken Asset Reference report as shown below:

You can select the report just now created by you in the drop down as shown below:

Provide scheduler expression as per your needs and select the result refresh interval for every 10s or as per your needs

You can see the results as shown below once the process is kicked off and it will also show the current row its processing and also CPU/Heap usage.

For some reason, if your system CPU/Heap is throttling then from the backend it takes care of not running your scheduler or you can also manually unschedule the scheduler.

Once your system’s CPU comes back to normal you can go back and select the report and schedule again and report generation picks from the current row where it was left off.

Once the processing is complete click on the report name to download the generated report.

Cross verification:

You can rerun the generated report on the MCP broken asset reference.
Generate the Splunk (logs) results by running a query to get all the assets to call (/content/dam) requests on dispatcher/publisher from the past 1year or so.
You can also reach out to the Analytics team, requesting image impressions (data on image usage) from the past 1 year or so.

Please provide your valuable feedback in the comments.

AEM Tool – Access CRX Package manager in AEM PROD

Problem statement:

How to access the CRX package manager in AEM PROD or AEM as Cloud services?

User cases:

Latest content package from PROD to lowers or local for debugging purposes
Install the content package on PROD
Continue the PROD deployment during CM outage in between deployment

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

AEM operations
Managing templates
Cloud configurations
ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Package handler can be used to upload, install, build or delete packages and we are using JCR Package manager to achieve all the above options.

Usually, if want to perform any operations on AEM as managed services are AEMaaCS we need to go through CAB, and AMS resources will perform all the operations as mentioned on the CAB. However, if your project has shared resources then all the priority package operations will take more time on PROD or any other environments.

Add the following Sling model fields as shown below to accept the inputs:

package com.aem.operations.core.models;

import java.io.InputStream;
import javax.inject.Named;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import com.adobe.acs.commons.mcp.form.FileUploadComponent;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.GeneratedDialog;
import com.adobe.acs.commons.mcp.form.RadioComponent;
import lombok.Getter;
import lombok.Setter;

@Model(adaptables = { Resource.class,SlingHttpServletRequest.class }, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class PackageHandlerModel extends GeneratedDialog {

    @Getter
    @Setter
    @Named(value = "inputPackage")
    @FormField(name = "Package", description = "Upload JCR Package", component = FileUploadComponent.class)
    private InputStream inputPackage = null;

    @Getter
    @Setter
    @Named(value = "packageName")
    @FormField(
            name = "Package Name",
            category = "General",
            required = false,
            description = "Enter the package name",
            hint = "Name")
    private String packageName;

    @Getter
    @Setter
    @Named(value = "packageGroup")
    @FormField(
            name = "Package Group",
            category = "General",
            required = false,
            description = "Enter the package group",
            hint = "Group")
    private String packageGroup;

    @Getter
    @Setter
    @Named(value = "packageVersion")
    @FormField(
            name = "Package Version",
            category = "General",
            required = false,
            description = "Enter the package version",
            hint = "Version")
    private String packageVersion;

    @Getter
    @Setter
    @Named(value = "packageOperation")
    @FormField(
            name = "Package Operation",
            description = "Select the operation to be performed",
            required = true,
            component = RadioComponent.EnumerationSelector.class,
            options = {"horizontal", "default=UPLOAD"})
    private Mode packageOperation;

    public enum Mode {
        UPLOAD, UPLOAD_INSTALL, BUILD, INSTALL, DELETE
    }
}

Add the following Servlet to process the request as shown below:

package com.aem.operations.core.servlets;

import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import com.aem.operations.core.models.PackageHandlerModel;
import com.aem.operations.core.services.PackageHandlerService;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.sling.servlets.post.JSONResponse;
import org.jetbrains.annotations.Nullable;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.granite.rest.Constants;
import com.google.gson.Gson;
import com.google.gson.JsonObject;

@Component(
        service = { Servlet.class },
        property = {
                "sling.servlet.paths=" + PackageHandlerServlet.RESOURCE_PATH,
                "sling.servlet.methods=POST"
        }
)
public class PackageHandlerServlet extends SlingAllMethodsServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger LOGGER = LoggerFactory.getLogger(PackageHandlerServlet.class);
    public static final String RESOURCE_PATH = "/bin/triggerPackageHandler";
    private static final String MESSAGE = "message";

    @Reference
    private PackageHandlerService packageHandlerService;

    @Override
    protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response) throws ServletException, IOException {
        // Set response headers.
        response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE);
        response.setCharacterEncoding(Constants.DEFAULT_CHARSET);

        JsonObject jsonResponse = new JsonObject();
        ResourceResolver resourceResolver = request.getResourceResolver();

        String packageName = request.getParameter("packageName");
        String packageGroup = request.getParameter("packageGroup");
        String packageVersion = request.getParameter("packageVersion");
        String packageOperation = request.getParameter("packageOperation");
        @Nullable RequestParameter inputPackStream = request.getRequestParameter("file");

        String packagePath = StringUtils.EMPTY;
        if (StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.UPLOAD.toString(), packageOperation)) {
            packagePath = packageHandlerService.uploadPack(resourceResolver, inputPackStream);
        } else  if(StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.BUILD.toString(), packageOperation)) {
            packagePath = packageHandlerService.buildPackage(resourceResolver, packageGroup, packageName, packageVersion);
        } else if(StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.UPLOAD_INSTALL.toString(), packageOperation)) {
            packagePath = packageHandlerService.uploadAndInstallPack(resourceResolver, inputPackStream);
        } else if(StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.INSTALL.toString(), packageOperation)) {
            packagePath = packageHandlerService.installPackage(resourceResolver, packageGroup, packageName, packageVersion, ImportMode.REPLACE, AccessControlHandling.IGNORE);
        } else if(StringUtils.equalsIgnoreCase(PackageHandlerModel.Mode.DELETE.toString(), packageOperation)) {
            packagePath = packageHandlerService.deletePackage(resourceResolver, packageGroup, packageName, packageVersion);
        }
        if(StringUtils.isNotEmpty(packagePath)) {
            jsonResponse.addProperty(MESSAGE, "Package Operation is Successful and path is <a href=\"" + packagePath + "\">" + packagePath + "</a>");
        } else {
            LOGGER.error("Unable to process package operation");
        }
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }
}

Add the following Package service as shown below:

package com.aem.operations.core.services;

import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.ResourceResolver;
import org.jetbrains.annotations.Nullable;

public interface PackageHandlerService {

	
	/**
	 * @param resourceResolver
	 * @param inputPackage
	 * @return package Path
	 */
	public String uploadPack(ResourceResolver resourceResolver, @Nullable RequestParameter inputPackage);

	/**
	 * @param resourceResolver
	 * @param groupName
	 * @param packageName
	 * @param version
	 * @return
	 */
	public String buildPackage(ResourceResolver resourceResolver, final String groupName, final String packageName,
			final String version);

	/**
	 * @param resourceResolver
	 * @param groupName
	 * @param packageName
	 * @param version
	 * @param importMode
	 * @param aclHandling
	 * @return package Path
	 */
	public String installPackage(ResourceResolver resourceResolver, final String groupName, final String packageName,
			final String version, final ImportMode importMode, final AccessControlHandling aclHandling);

	/**
	 * @param resourceResolver
	 * @param inputPackage
	 * @return package Path
	 */
	public String uploadAndInstallPack(ResourceResolver resourceResolver, @Nullable RequestParameter inputPackage);

	/**
	 * @param resourceResolver
	 * @param groupName
	 * @param packageName
	 * @param version
	 * @return package Path
	 */
	public String deletePackage(ResourceResolver resourceResolver, final String groupName, final String packageName,
			final String version);
}

Add the following Package service Implementation as shown below:

package com.aem.operations.core.services.impl;

import com.aem.operations.core.services.PackageHandlerService;
import com.aem.operations.core.utils.VltUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.fs.io.ImportOptions;
import org.apache.jackrabbit.vault.packaging.*;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.ResourceResolver;
import org.jetbrains.annotations.Nullable;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import java.io.IOException;
import java.io.InputStream;

@Component(service = PackageHandlerService.class, immediate = true, name = "Package Handler Service")
public class PackageHandlerServiceImpl implements PackageHandlerService {

	private static final Logger LOGGER = LoggerFactory.getLogger(PackageHandlerServiceImpl.class);

	@Reference
	private Packaging packaging;

	@Override
	public String uploadPack(ResourceResolver resourceResolver, @Nullable RequestParameter inputPackage) {
		if (null != inputPackage) {
			try (InputStream inputPack = inputPackage.getInputStream()) {
				if (null != inputPack) {
					final JcrPackageManager packageManager = packaging
							.getPackageManager(resourceResolver.adaptTo(Session.class));
					JcrPackage jcrPackage = packageManager.upload(inputPack, true);
					return jcrPackage.getNode().getPath();
				}
			} catch (IOException | RepositoryException e) {
				LOGGER.error("Could not upload package {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

	@Override
	public String buildPackage(ResourceResolver resourceResolver, String groupName, String packageName,
			String version) {
		if (StringUtils.isNotEmpty(packageName) && StringUtils.isNotEmpty(groupName)) {
			final JcrPackageManager packageManager = packaging
					.getPackageManager(resourceResolver.adaptTo(Session.class));
			final PackageId packageId = new PackageId(groupName, packageName, version);
			try (JcrPackage jcrPackage = packageManager.open(packageId)) {
				if(null != jcrPackage) {
					packageManager.assemble(jcrPackage, null);
					return jcrPackage.getNode().getPath();
				}
			} catch (RepositoryException | PackageException | IOException e) {
				LOGGER.error("Could not build package {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

	@Override
	public String installPackage(ResourceResolver resourceResolver, String groupName, String packageName,
			String version, ImportMode importMode, AccessControlHandling aclHandling) {
		if (StringUtils.isNotEmpty(packageName) && StringUtils.isNotEmpty(groupName)) {
			final JcrPackageManager packageManager = packaging
					.getPackageManager(resourceResolver.adaptTo(Session.class));
			final PackageId packageId = new PackageId(groupName, packageName, version);
			try (JcrPackage jcrPackage = packageManager.open(packageId)) {
				if(null != jcrPackage) {
					final ImportOptions opts = VltUtils.getImportOptions(aclHandling, importMode);
					jcrPackage.install(opts);
					return jcrPackage.getNode().getPath();
				}
			} catch (RepositoryException | PackageException | IOException e) {
				LOGGER.error("Could not install package {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

	@Override
	public String uploadAndInstallPack(ResourceResolver resourceResolver, @Nullable RequestParameter inputPackage) {
		if (null != inputPackage) {
			try (InputStream inputPack = inputPackage.getInputStream()) {
				if (null != inputPack) {
					final JcrPackageManager packageManager = packaging
							.getPackageManager(resourceResolver.adaptTo(Session.class));
					try (JcrPackage jcrPackage = packageManager.upload(inputPack, true)) {
						installPackage(jcrPackage, ImportMode.REPLACE, AccessControlHandling.IGNORE);
						return jcrPackage.getNode().getPath();
					} catch (RepositoryException e) {
						LOGGER.error("Could not Upload and Install package due to Repository Exception {}", e.getMessage());
						e.printStackTrace();
					}
				}
			} catch (IOException e) {
				LOGGER.error("Could not Upload and Install package due to IO Exception {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

	public String installPackage(JcrPackage jcrPackage, final ImportMode importMode,
			final AccessControlHandling aclHandling) {
		try {
			final ImportOptions opts = VltUtils.getImportOptions(aclHandling, importMode);
			jcrPackage.install(opts);
			return jcrPackage.getNode().getPath();
		} catch (RepositoryException | PackageException | IOException e) {
			LOGGER.error("Could not install built package {}", e.getMessage());
			return StringUtils.EMPTY;
		}
	}

	@Override
	public String deletePackage(ResourceResolver resourceResolver, String groupName, String packageName,
			String version) {
		if (StringUtils.isNotEmpty(packageName) && StringUtils.isNotEmpty(groupName)) {
			final JcrPackageManager packageManager = packaging
					.getPackageManager(resourceResolver.adaptTo(Session.class));
			final PackageId packageId = new PackageId(groupName, packageName, version);
			try (JcrPackage jcrPackage = packageManager.open(packageId)) {
				if(null != jcrPackage) {
					String path = jcrPackage.getNode().getPath();
					packageManager.remove(jcrPackage);
					return path;
				}				
			} catch (RepositoryException e) {
				LOGGER.error("Could not delete package {}", e.getMessage());
				return StringUtils.EMPTY;
			}
		}
		return StringUtils.EMPTY;
	}

}

Add the following VltUtils as shown below:

package com.aem.operations.core.utils;

import org.apache.jackrabbit.vault.fs.api.ImportMode;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.fs.io.ImportOptions;

/**
 * Utility class for creating vlt filters and import/export options
 */
public class VltUtils {

    public static ImportOptions getImportOptions(AccessControlHandling aclHandling, ImportMode importMode) {
        ImportOptions opts = new ImportOptions();
        if (aclHandling != null) {
            opts.setAccessControlHandling(aclHandling);
        } else {
            // default to overwrite
            opts.setAccessControlHandling(AccessControlHandling.OVERWRITE);
        }
        if (importMode != null) {
            opts.setImportMode(importMode);
        } else {
            // default to update
            opts.setImportMode(ImportMode.UPDATE);
        }

        return opts;
    }
}

Add the following HTL to render the page:

<!DOCTYPE html>
<head>
   <link rel="shortcut icon" href="/libs/granite/core/content/login/favicon.ico">
</head>
<body class="coral--light foundation-layout-util-maximized-alt">
   <sly data-sly-call="${clientLib.all @ categories=['coralui3','granite.ui.coral.foundation','granite.ui.shell','cq.authoring.dialog', 'packagehandelr.base']}"
      data-sly-use.clientLib="/libs/granite/sightly/templates/clientlib.html"/>
   <coral-shell-header aria-hidden="false" aria-label="Header Bar"
      class="coral--dark granite-shell-header coral3-Shell-header"
      role="region">
      <coral-shell-header-home aria-level="2" class="globalnav-toggle" data-globalnav-toggle-href="/" role="heading">
         <a class="coral3-Shell-homeAnchor" href="/"
            icon="adobeExperienceManagerColor" is="coral-shell-homeanchor"
            style="display: inline-block; padding-right: 0;">
            <coral-icon
               aria-label="adobe experience manager color"
               class="coral3-Icon coral3-Shell-homeAnchor-icon coral3-Icon--adobeExperienceManagerColor coral3-Icon--sizeM"
               icon="adobeExperienceManagerColor" role="img"
               size="M"></coral-icon>
            <coral-shell-homeanchor-label>Adobe Experience Manager
            </coral-shell-homeanchor-label>
         </a>
         <span style="line-height: 2.375rem;">/ AEM Operations / ${properties.jcr:title} </span>
      </coral-shell-header-home>
   </coral-shell-header>
   <div class="foundation-layout-panel">
      <div class="foundation-layout-panel-header">
         <betty-titlebar>
            <betty-titlebar-title>
               <span aria-level="1" class="granite-title" role="heading">${properties.jcr:title}</span>
            </betty-titlebar-title>
            <betty-titlebar-primary></betty-titlebar-primary>
            <betty-titlebar-secondary>
            </betty-titlebar-secondary>
         </betty-titlebar>
      </div>
      <div class="content-container-inner coral-Well" style="width:60%; margin:0 auto;">
         <sly data-sly-use.packprops="com.aem.operations.core.models.PackageHandlerModel">
            <sly data-sly-resource="${packprops.formResource}"/>
         </sly>
         <coral-actionbar-item class="coral3-ActionBar-item">
            <button class="coral3-Button coral3-Button--primary package-initiator" icon="checkCircle" iconsize="S"
               is="coral-button"
               size="M" variant="primary">
               <coral-icon alt="" class="coral3-Icon coral3-Icon--sizeS coral3-Icon--checkCircle" icon="engagement"
                  size="S"></coral-icon>
               <coral-button-label>
                  Start the Process
               </coral-button-label>
            </button>
         </coral-actionbar-item>
          <!-- The Modal -->
          <div id="popup-modal" class="modal">
              <!-- Modal content -->
              <div class="modal-content">
                  <span class="close">&times;</span>
                  <span class="loading">Loading</span>
                  <span class="loader"></span>
              </div>
          </div>
      </div>
   </div>
</body>

Add the following JS code to process the ajax request as shown below:

var Coral = window.Coral || {},
    Granite = window.Granite || {};

(function (window, document, $, Coral) {
    "use strict";
    var packageOperation = "UPLOAD";
    $(document).on("foundation-contentloaded", function (e) {

        var SITE_PATH = "/conf/aemoperations/settings/tools/packagehandler-initiator.html",
            ui = $(window).adaptTo("foundation-ui");

        if (window.location.href.indexOf(SITE_PATH) < 0) {
            return;
        }

        if (packageOperation === "UPLOAD") {
            $("input[name='./packageName']").attr("disabled", true);
            $("input[name='./packageGroup']").attr("disabled", true);
            $("input[name='./packageVersion']").attr("disabled", true);
        }

        $(document).off("change", ".coral-RadioGroup").on("change", ".coral-RadioGroup", function (event) {
            packageOperation = event.target.value;
			if (packageOperation === "UPLOAD" || packageOperation === "UPLOAD_INSTALL") {
				$("input[name='./packageName']").attr("disabled", true);
				$("input[name='./packageGroup']").attr("disabled", true);
				$("input[name='./packageVersion']").attr("disabled", true);
				$("coral-fileupload[name='./inputPackage']").attr("disabled", false);
			} else if(packageOperation === "BUILD" || packageOperation === "INSTALL" || packageOperation === "DELETE"){
				$("input[name='./packageName']").attr("disabled", false);
				$("input[name='./packageGroup']").attr("disabled", false);
				$("input[name='./packageVersion']").attr("disabled", false);
				$("coral-fileupload[name='./inputPackage']").attr("disabled", true);
			}
        });

        $(document).off("change", ".close").on("change", ".close", function (event) {
            $(".modal").hide();
        });

        function getValueByName(fieldName, isMandatory) {
            var fieldValue = ($("input[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("input[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("input[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getFileByName(fieldName) {
            var fieldInput = $("input[name='" + fieldName + "']");
            if(null != fieldInput && null != fieldInput[0]){
                return fieldInput[0].files[0];
            }
        }

        $(document).off("click", ".package-initiator").on("click", ".package-initiator", function (event) {
            event.preventDefault();

            var packageName = getValueByName('./packageName', false),
                packageGroup = getValueByName('./packageGroup', false),
                packageVersion = getValueByName('./packageVersion', false),
                inputPackage = getFileByName('./inputPackage');

            if (!packageOperation) {
                return;
            }

            var formData = new FormData();
            if(null != inputPackage){
                formData.append("file", inputPackage, inputPackage.name);
            }
            formData.append("packageName", packageName);
            formData.append("packageGroup", packageGroup);
            formData.append("packageVersion", packageVersion);
            formData.append("packageOperation", packageOperation);

			$(".loading").html("PLEASE WAIT "+packageOperation+"ING PACKAGE");
			$(".modal").show();

            $.ajax({
                url: "/bin/triggerPackageHandler",
                method: "POST",
                async: true,
                cache: false,
                contentType: false,
                processData: false,
                data: formData
            }).done(function (data) {
                if (data && data.message){
                    ui.notify("Success", data.message, "success");
                    var dialog = new Coral.Dialog();
                    dialog.id = 'dialogSuccess';
                    dialog.header.innerHTML = 'Success';
                    dialog.content.innerHTML = data.message;
                    dialog.footer.innerHTML = '<button class="ok-button" is="coral-button" variant="primary" icon="check" coral-close>OK</button>';
                    dialog.variant = 'success';
                    dialog.closable = "on";
                    dialog.show();
					$(".modal").hide();
                }else{
                    ui.notify("Error", "Unable to process package operation", "error");
                }
            }).fail(function (data) {
				$(".modal").hide();
                if (data && data.responseJSON && data.responseJSON.message){
                    ui.notify("Error", data.responseJSON.message, "error");
                }else{
                    ui.notify("Error", "Unable to process package operation", "error");
                }
            });
        });
    });
})(window, document, $, Coral);

Once the code is built and deployed you will be able to access the tool by navigating the:

Tools -> {Tool Section} -> {Tool Name}

Now you can access the package handler and upload the packages to AEM as an upload or upload and install.

If you’re building any existing packages then you can build, install or delete packages. Provide package name, version, and group details to pull the package.

Provide the desired package name and Description and you can see the package path will be prompted and you will be able to download the built package by clicking on the link:

Working Code:
You can also access the working code on my GitHub repository link: https://github.com/kiransg89/aemoperations

Note: You can access this package over domain on AEM / AEMaaCS and over any environments and architects can handle the permissions by adding appropriate rep:policy on cq generator node or conf page

AEM Tool – Publish / UnPublish / Delete List of pages

Problem statement:

Can we Publish / Un Publish / Delete the list of paths mentioned in an excel sheet? Or provided has a linefeed

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

AEM operations
Managing templates
Cloud configurations
ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Usually, product owners or authors would like to Publish certain pages like offer/product/article pages based on business requirements and also would like to unpublish and delete to clean up the pages which are unnecessary.

This process will be really helpful during excel sheet based migration.

The list Replication process usually validates the paths and if it exists then activates or deactivates or deletes pages or content paths. It accepts an excel sheet or list of paths as a line feed.

You can also provide a list of agents to be activated/deactivated to for example only to Brightspot or Brightcove connector.

In order to create a tool please use the Tool generator to give your tool name and description: https://kiransg.com/2022/11/24/aem-tool-create-generate-tool-from-scratch

Add the following Sling model fields as shown below to accept the inputs:

package com.aem.operations.core.models;

import com.adobe.acs.commons.mcp.form.*;
import lombok.Getter;
import lombok.Setter;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import javax.inject.Named;
import java.io.InputStream;

@Model(adaptables = { Resource.class,SlingHttpServletRequest.class }, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class ListTreeActivationModel extends GeneratedDialog {

    public enum QueueMethod {
        USE_MCP_QUEUE, USE_PUBLISH_QUEUE, MCP_AFTER_10K
    }

    public enum ReplicationAction {
        PUBLISH, UNPUBLISH, DELETE
    }

    @Getter
    @Setter
    @Named(value = "repPathExcel")
    @FormField(name = "Replication Excel", description = "Upload Replication Excel", component = FileUploadComponent.class)
    private InputStream repPathExcel = null;

    @Getter
    @Setter
    @Named(value = "queueMethod")
    @FormField(
            name = "Queueing Method",
            description = "For small publishing tasks, standard is sufficient.  For large folder trees, MCP is recommended.",
            required = true,
            component = SelectComponent.EnumerationSelector.class,
            options = {"horizontal", "default=USE_MCP_QUEUE"})
    QueueMethod queueMethod = QueueMethod.USE_MCP_QUEUE;

    @Getter
    @Setter
    @Named(value = "agents")
    @FormField(name = "Agents",
            component = TextfieldComponent.class,
            hint = "(leave blank for default agents)",
            description = "Publish agents to use, if blank then all default agents will be used. Multiple agents can be listed using commas or regex.")
    private String agents;

    @Getter
    @Setter
    @Named(value = "reAction")
    @FormField(name = "Action",
            component = SelectComponent.EnumerationSelector.class,
            description = "Publish or Unpublish?",
            options = "default=PUBLISH")
    ReplicationAction reAction = ReplicationAction.PUBLISH;
}

Add the following Servlet to process the request as shown below:

package com.aem.operations.core.servlets;

import java.io.IOException;
import java.io.PrintWriter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
import javax.jcr.Session;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import com.aem.operations.core.models.ListTreeActivationModel;
import com.aem.operations.core.utils.RetryUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.sling.servlets.post.JSONResponse;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.acs.commons.data.CompositeVariant;
import com.adobe.acs.commons.data.Spreadsheet;
import com.adobe.granite.rest.Constants;
import com.day.cq.replication.AgentFilter;
import com.day.cq.replication.ReplicationActionType;
import com.day.cq.replication.ReplicationException;
import com.day.cq.replication.ReplicationOptions;
import com.day.cq.replication.Replicator;
import com.google.gson.Gson;
import com.google.gson.JsonObject;

@Component(service = { Servlet.class }, property = { "sling.servlet.paths=" + ListActiavationServlet.RESOURCE_PATH,
        "sling.servlet.methods=POST" })
public class ListActiavationServlet extends SlingAllMethodsServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger LOGGER = LoggerFactory.getLogger(ListActiavationServlet.class);
    public static final String RESOURCE_PATH = "/bin/triggerListActivation";
    private static final String MESSAGE = "message";
    private static final String DESTINATION_PATH = "destination";
    private static int ASYNC_LIMIT = 10000;


    private List<String> agentList = new ArrayList<>();
    private Spreadsheet spreadsheet;
    AtomicInteger replicationCount = new AtomicInteger();

    @Reference
    Replicator replicator;

    @Override
    protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response)
            throws ServletException, IOException {

        // Set response headers.
        response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE);
        response.setCharacterEncoding(Constants.DEFAULT_CHARSET);

        JsonObject jsonResponse = new JsonObject();

        String queueMethod = request.getParameter("queueMethod");
        String agents = request.getParameter("agents");
        String reAction = request.getParameter("reAction");
        String pathsList = request.getParameter("pathsList");

        @Nullable
        RequestParameter inputPackStream = request.getRequestParameter("file");
        if(inputPackStream != null) {
            spreadsheet = new Spreadsheet(inputPackStream, DESTINATION_PATH).buildSpreadsheet();
        }
        if (null != spreadsheet) {
            AgentFilter replicationAgentFilter = prepareAgents(agents);
            if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.PUBLISH.toString(), reAction)) {
                activateTreeStructure(request.getResourceResolver(), queueMethod, reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Activated all the paths provided in Excel sheet");
            } else if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.UNPUBLISH.toString(), reAction)) {
                deactivateTreeStructure(request.getResourceResolver(), reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deactivated all the paths provided in Excel sheet");
            } else {
                deleteTree(request.getResourceResolver());
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deleted all the paths provided in Excel sheet");
            }
        } else if(StringUtils.isNotEmpty(pathsList)){
            AgentFilter replicationAgentFilter = prepareAgents(agents);
            Set<String> pages = new HashSet<>(Arrays.asList(StringUtils.split(pathsList, '\n'))).stream().map(String::trim).collect(Collectors.toSet());
            if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.PUBLISH.toString(), reAction)) {
                activateTreeStructure(request.getResourceResolver(), pages, queueMethod, reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Activated all the paths provided in Excel sheet");
            } else if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.UNPUBLISH.toString(), reAction)) {
                deactivateTreeStructure(request.getResourceResolver(), pages, reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deactivated all the paths provided in Line Feed");
            } else {
                deleteTree(request.getResourceResolver(), pages);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deleted all the paths provided in Line Feed");
            }
        } else {
            jsonResponse.addProperty(MESSAGE, "Unable to process Activation request because of invalid Inputs");
        }
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }

    private AgentFilter prepareAgents(String agents) {
        AgentFilter replicationAgentFilter;
        if (StringUtils.isEmpty(agents)) {
            replicationAgentFilter = AgentFilter.DEFAULT;
        } else {
            agentList = Arrays.asList(agents.toLowerCase(Locale.ENGLISH).split(","));
            replicationAgentFilter = agent -> agentList.stream()
                    .anyMatch(p -> p.matches(agent.getId().toLowerCase(Locale.ENGLISH)));
        }
        return replicationAgentFilter;
    }

    private void activateTreeStructure(@NotNull ResourceResolver resourceResolver, String queueMethod, String reAction, AgentFilter replicationAgentFilter) {
        spreadsheet.getDataRowsAsCompositeVariants()
                .forEach(row -> performReplication(resourceResolver, getString(row, DESTINATION_PATH), queueMethod, reAction, replicationAgentFilter));
    }

    private void deactivateTreeStructure(@NotNull ResourceResolver resourceResolver, String reAction, AgentFilter replicationAgentFilter) {
        spreadsheet.getDataRowsAsCompositeVariants()
                .forEach(row -> performAsynchronousReplication(resourceResolver, getString(row, DESTINATION_PATH), reAction, replicationAgentFilter));
    }

    private void activateTreeStructure(@NotNull ResourceResolver resourceResolver, Set<String> pages, String queueMethod, String reAction, AgentFilter replicationAgentFilter) {
        pages.stream().forEach(path -> performReplication(resourceResolver, path, queueMethod, reAction, replicationAgentFilter));
    }

    private void deactivateTreeStructure(@NotNull ResourceResolver resourceResolver, Set<String> pages, String reAction, AgentFilter replicationAgentFilter) {
        pages.stream().forEach(path -> performAsynchronousReplication(resourceResolver, path, reAction, replicationAgentFilter));
    }

    private void performReplication(@NotNull ResourceResolver resourceResolver, String path, String queueMethod, String reAction, AgentFilter replicationAgentFilter) {
        int counter = replicationCount.incrementAndGet();
        if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.QueueMethod.USE_MCP_QUEUE.toString(), queueMethod)
                || (StringUtils.equalsIgnoreCase(ListTreeActivationModel.QueueMethod.MCP_AFTER_10K.toString(),
                queueMethod) && counter >= ASYNC_LIMIT)) {
            performSynchronousReplication(resourceResolver, path, reAction, replicationAgentFilter);
        } else {
            performAsynchronousReplication(resourceResolver, path, reAction, replicationAgentFilter);
        }
    }

    private void performSynchronousReplication(@NotNull ResourceResolver resourceResolver, String path, String reAction, AgentFilter replicationAgentFilter) {
        ReplicationOptions options = buildOptions(replicationAgentFilter);
        options.setSynchronous(true);
        scheduleReplication(resourceResolver, options, path, reAction);
    }

    private void performAsynchronousReplication(@NotNull ResourceResolver resourceResolver, String path, String reAction, AgentFilter replicationAgentFilter) {
        ReplicationOptions options = buildOptions(replicationAgentFilter);
        options.setSynchronous(false);
        scheduleReplication(resourceResolver, options, path, reAction);
    }

    private void scheduleReplication(@NotNull ResourceResolver resourceResolver, ReplicationOptions options,
                                     String path, String reAction) {
        Session session = resourceResolver.adaptTo(Session.class);
        boolean isPublish = StringUtils
                .equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.PUBLISH.toString(), reAction);
        try {
            replicator.replicate(session,
                    isPublish ? ReplicationActionType.ACTIVATE : ReplicationActionType.DEACTIVATE, path, options);
        } catch (ReplicationException e) {
            LOGGER.error("Error occured during replication {}", e.getMessage());
        }
    }

    @SuppressWarnings("rawtypes")
    private void deleteTree(ResourceResolver resourceResolver) {
        try {
            RetryUtils.withRetry(30, 500, () -> {
                Iterator<Map<String, CompositeVariant>> spredIterator = spreadsheet.getDataRowsAsCompositeVariants()
                        .iterator();
                while (spredIterator.hasNext()) {
                    Map<String, CompositeVariant> row = spredIterator.next();
                    deleteContent(resourceResolver, getString(row, DESTINATION_PATH));
                }
                commitChanges(resourceResolver);
            });
        } catch (Exception e) {
            LOGGER.error("Error occured during Retrying {}", e.getMessage());
        }
    }

    private void deleteTree(ResourceResolver resourceResolver, Set<String> pages) {
        try {
            RetryUtils.withRetry(30, 500, () -> {
                pages.forEach(path -> deleteContent(resourceResolver, path));
                commitChanges(resourceResolver);
            });
        } catch (Exception e) {
            LOGGER.error("Error occured during Retrying {}", e.getMessage());
        }
    }

    private void deleteContent(ResourceResolver resourceResolver, String destinationPath) {
        try {
            @Nullable
            Resource destinationResource = resourceResolver.getResource(destinationPath);
            if (null != destinationResource) {
                resourceResolver.delete(destinationResource);
            }
        } catch (PersistenceException e) {
            LOGGER.error("unable to delete content {}", e.getMessage());
        }
    }

    private void commitChanges(ResourceResolver resourceResolver) throws PersistenceException {
        if (resourceResolver.hasChanges()) {
            resourceResolver.commit();
            resourceResolver.refresh();
        }
    }

    private ReplicationOptions buildOptions(AgentFilter replicationAgentFilter) {
        ReplicationOptions options = new ReplicationOptions();
        options.setFilter(replicationAgentFilter);
        return options;
    }

    @SuppressWarnings({ "rawtypes", "unchecked" })
    private String getString(Map<String, CompositeVariant> row, String path) {
        CompositeVariant v = row.get(path.toLowerCase(Locale.ENGLISH));
        if (v != null) {
            return (String) v.getValueAs(String.class);
        } else {
            return null;
        }
    }
}

Add the following Retry Utils as shown below:

package com.aem.operations.core.utils;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryUtils {
    private static final Logger log = LoggerFactory.getLogger(RetryUtils.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int i = 0; i < maxTimes; i++) {
            try {
                call.process();
                return true;
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (i + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

Add the following HTL to render the page:

<!DOCTYPE html>
<head>
   <link rel="shortcut icon" href="/libs/granite/core/content/login/favicon.ico">
</head>
<body class="coral--light foundation-layout-util-maximized-alt">
   <sly data-sly-call="${clientLib.all @ categories=['coralui3','granite.ui.coral.foundation','granite.ui.shell','cq.authoring.dialog', 'listreplication.base']}"
      data-sly-use.clientLib="/libs/granite/sightly/templates/clientlib.html"/>
   <coral-shell-header aria-hidden="false" aria-label="Header Bar"
      class="coral--dark granite-shell-header coral3-Shell-header"
      role="region">
      <coral-shell-header-home aria-level="2" class="globalnav-toggle" data-globalnav-toggle-href="/" role="heading">
         <a class="coral3-Shell-homeAnchor" href="/"
            icon="adobeExperienceManagerColor" is="coral-shell-homeanchor"
            style="display: inline-block; padding-right: 0;">
            <coral-icon
               aria-label="adobe experience manager color"
               class="coral3-Icon coral3-Shell-homeAnchor-icon coral3-Icon--adobeExperienceManagerColor coral3-Icon--sizeM"
               icon="adobeExperienceManagerColor" role="img"
               size="M"></coral-icon>
            <coral-shell-homeanchor-label>Adobe Experience Manager
            </coral-shell-homeanchor-label>
         </a>
         <span style="line-height: 2.375rem;">/ AEM Operations / ${properties.jcr:title} </span>
      </coral-shell-header-home>
   </coral-shell-header>
   <div class="foundation-layout-panel">
      <div class="foundation-layout-panel-header">
         <betty-titlebar>
            <betty-titlebar-title>
               <span aria-level="1" class="granite-title" role="heading">${properties.jcr:title}</span>
            </betty-titlebar-title>
            <betty-titlebar-primary></betty-titlebar-primary>
            <betty-titlebar-secondary>
            </betty-titlebar-secondary>
         </betty-titlebar>
      </div>
      <div class="content-container-inner coral-Well" style="width:60%; margin:0 auto;">
         <div class="acsCommons-System-Notifications-Form-row">
		   <label class="coral-Form-fieldlabel">List of Paths</label>
		   <label> Note: Add the list of paths has a Line feed (No need for comma separation)</label>
		   <textarea
		      class="coral-Textfield coral-Textfield--multiline acsCommons-System-Notifications-Page-input--textarea"
		      rows="6"
		      name="./pathsList"></textarea>
		 </div>
		 <label> Note: Create a Excel Sheet with column called <b>destination</b> and the list of values and upload</label>
         <sly data-sly-use.packprops="com.aem.operations.core.models.ListTreeActivationModel">
            <sly data-sly-resource="${packprops.formResource}"/>
         </sly>
         <coral-actionbar-item class="coral3-ActionBar-item">
            <button class="coral3-Button coral3-Button--primary activation-initiator" icon="checkCircle" iconsize="S"
               is="coral-button"
               size="M" variant="primary">
               <coral-icon alt="" class="coral3-Icon coral3-Icon--sizeS coral3-Icon--checkCircle" icon="publish"
                  size="S"></coral-icon>
               <coral-button-label>
                  Start the Replication
               </coral-button-label>
            </button>
         </coral-actionbar-item>
          <!-- The Modal -->
          <div id="popup-modal" class="modal">
              <!-- Modal content -->
              <div class="modal-content">
                  <span class="close">&times;</span>
                  <span class="loading">Loading</span>
                  <span class="loader"></span>
              </div>
          </div>
      </div>
   </div>
</body>

Add the following JS code to process the ajax request as shown below:

var Coral = window.Coral || {},
    Granite = window.Granite || {};

(function (window, document, $, Coral) {
    "use strict";
    $(document).on("foundation-contentloaded", function (e) {

        var SITE_PATH = "/conf/aemoperations/settings/tools/listreplication-initiator.html",
            ui = $(window).adaptTo("foundation-ui");

        if (window.location.href.indexOf(SITE_PATH) < 0) {
            return;
        }

        $(document).off("change", ".close").on("change", ".close", function (event) {
            $(".modal").hide();
        });

        function getValueByName(fieldName, isMandatory) {
            var fieldValue = ($("input[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("input[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("input[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getValueByNameArea(fieldName, isMandatory) {
            var fieldValue = ($("textarea[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("textarea[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("textarea[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getFileByName(fieldName) {
            var fieldInput = $("input[name='" + fieldName + "']");
            if(null != fieldInput && null != fieldInput[0]){
                return fieldInput[0].files[0];
            }
        }

        $(document).off("click", ".activation-initiator").on("click", ".activation-initiator", function (event) {
            event.preventDefault();

            var queueMethod = getValueByName('./queueMethod', false),
                agents = getValueByName('./agents', false),
                reAction = getValueByName('./reAction', false),
                pathsList = getValueByNameArea('./pathsList', true),
                repPathExcel = getFileByName('./repPathExcel');

            var formData = new FormData();
            if(null != repPathExcel){
                formData.append("file", repPathExcel, repPathExcel.name);
            }
            formData.append("queueMethod", queueMethod);
            formData.append("agents", agents);
            formData.append("reAction", reAction);
            formData.append("pathsList", pathsList);

			$(".loading").html("Replication in progress");
            $(".modal").show();

            $.ajax({
                url: "/bin/triggerListActivation",
                method: "POST",
                async: true,
                cache: false,
                contentType: false,
                processData: false,
                data: formData
            }).done(function (data) {
                if (data && data.message){
                    ui.notify("Success", data.message, "success");
                    var dialog = new Coral.Dialog();
                    dialog.id = 'dialogSuccess';
                    dialog.header.innerHTML = 'Success';
                    dialog.content.innerHTML = data.message;
                    dialog.footer.innerHTML = '<button class="ok-button" is="coral-button" variant="primary" icon="check" coral-close>OK</button>';
                    dialog.variant = 'success';
                    dialog.closable = "on";
                    dialog.show();
                    $(".modal").hide();
                }else{
                    ui.notify("Error", "Unable to process List Activation", "error");
                }
            }).fail(function (data) {
				$(".modal").hide();
                if (data && data.responseJSON && data.responseJSON.message){
                    ui.notify("Error", data.responseJSON.message, "error");
                }else{
                    ui.notify("Error", "Unable to process List Activation", "error");
                }
            });
        });
    });
})(window, document, $, Coral);

Once the code is built and deployed you will be able to access the tool by navigating the:

Tools -> {Tool Section} -> {Tool Name}

Now you can input the page paths as a line feed (no need to separate the paths using commas or any characters)

Upload the excel sheet or paths and select the queue method it’s recommended to use MCP Queue if you are activating more than 10K pages then it’s recommended to select MCP After 10K.

Select the Action to be performed.

Working Code:
You can also access the working code on my GitHub repository link: https://github.com/kiransg89/aemoperations

Note: You can access this package over domain on AEM / AEMaaCS and over any environments and architects can handle the permissions by adding appropriate rep:policy on cq generator node or conf page

AEM Tool – Perfect Content backup Package in AEM

Problem statement:

Create a perfect content backup package in AEM, whenever we want to create any package in AEM we provide the content paths in the package filter manually and what do we get?

We get only content pages.
What about images and referenced pages?
What about header/footer experience fragment pages or XF page-related context hub variations?

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

AEM operations
Managing templates
Cloud configurations
ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Perfect packager pulls the current page resource and iterates on each node and gets the referenced images, pages, and XF’s. Once all the paths are pulled it reiterates to find all the referenced pages to get all the images, XF’s, and pages.

Add the following Sling model fields as shown below to accept the inputs:

package com.aem.operations.core.models;

import javax.inject.Named;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.GeneratedDialog;
import lombok.Getter;
import lombok.Setter;

@Model(adaptables = { Resource.class,SlingHttpServletRequest.class }, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class PerfectPackageModel extends GeneratedDialog {

    @Getter
    @Setter
    @Named(value = "packageName")
    @FormField(
            name = "Package Name",
            category = "General",
            required = true,
            description = "Enter the package name",
            hint = "Name")
    private String packageName;

    @Getter
    @Setter
    @Named(value = "packageDescription")
    @FormField(
            name = "Package Description",
            category = "General",
            required = true,
            description = "Enter the package Description",
            hint = "Description")
    private String packageDescription;
}

Add the following Servlet to process the request as shown below:

package com.aem.operations.core.servlets;

import java.io.IOException;
import java.io.PrintWriter;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import com.aem.operations.core.utils.ReportRetryUtils;
import com.aem.operations.core.visitors.ContentVisitor;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.packaging.JcrPackage;
import org.apache.jackrabbit.vault.packaging.JcrPackageDefinition;
import org.apache.jackrabbit.vault.packaging.JcrPackageManager;
import org.apache.jackrabbit.vault.packaging.PackageException;
import org.apache.jackrabbit.vault.packaging.Packaging;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.sling.servlets.post.JSONResponse;
import org.jetbrains.annotations.NotNull;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.acs.commons.packaging.PackageHelper;
import com.adobe.granite.rest.Constants;
import com.google.gson.Gson;
import com.google.gson.JsonObject;

@Component(
        service = { Servlet.class },
        property = {
                "sling.servlet.paths=" + PerfectPackageServlet.RESOURCE_PATH,
                "sling.servlet.methods=POST"
        }
)
public class PerfectPackageServlet extends SlingAllMethodsServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger LOGGER = LoggerFactory.getLogger(PerfectPackageServlet.class);
    public static final String RESOURCE_PATH = "/bin/triggerPerfectPackage";
    private static final String MESSAGE = "message";

    private static final String DEFUALT_GROUP_NAME = "my_packages";
    private static final String DEFUALT_VERSION = "1.0";
    private static final String QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH = "/apps/acs-commons/components/utilities/packager/query-packager/definition/package-thumbnail.png";

    @Reference
    private PackageHelper packageHelper;

    @Reference
    private Packaging packaging;

    @Override
    protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response) throws ServletException, IOException {
        // Set response headers.
        response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE);
        response.setCharacterEncoding(Constants.DEFAULT_CHARSET);

        JsonObject jsonResponse = new JsonObject();

        String packageName = request.getParameter("packageName");
        String packageDescription = request.getParameter("packageGroup");
        String pathsList = request.getParameter("pathsList");

        String packagePath = StringUtils.EMPTY;
        if(StringUtils.isNotEmpty(pathsList)) {
            packagePath = collectReferences(request.getResourceResolver(), pathsList, packageName, packageDescription);
        }
        if(StringUtils.isNotEmpty(packagePath)) {
            jsonResponse.addProperty(MESSAGE, "Package creation is Successful and path is <a href=\"" + packagePath + "\">" + packagePath + "</a>");
        } else {
            LOGGER.error("Unable to process package creation");
        }
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }

    private String collectReferences(ResourceResolver resourceResolver, String pagePaths, String packageName, String packageDescription) {
        Set<String> contentPaths = new HashSet<>();
        Set<String> xfPaths = new HashSet<>();
        Set<String> damPaths = new HashSet<>();
        ContentVisitor contentVisitor =  new ContentVisitor();
        Set<String> pages = new HashSet<>(Arrays.asList(StringUtils.split(pagePaths, '\n'))).stream().map(r -> r.trim() + "/jcr:content").collect(Collectors.toSet());
        try {
            iterateContentPaths(resourceResolver, contentPaths, xfPaths, damPaths, contentVisitor, pages);
            damPaths.addAll(iterateContent(resourceResolver, contentVisitor, damPaths));
            ReportRetryUtils.withRetry(2, 300, () -> xfPaths.addAll(iterateContent(resourceResolver, contentVisitor, xfPaths)));
            Set<String> allPaths = Stream.of(pages, contentPaths, xfPaths, damPaths).flatMap(Collection::stream).collect(Collectors.toSet());
            return packagePaths(resourceResolver, allPaths, packageName, packageDescription);
        } catch (Exception e) {
            LOGGER.error("Exception occurred during package creation in Retry {}", e.getMessage());
        }
        return StringUtils.EMPTY;
    }

    private Set<String> iterateContent(ResourceResolver resourceResolver, ContentVisitor contentVisitor, Set<String> pages) {
        Set<String> interimPaths = new HashSet<>();
        for(String page : pages) {
            contentVisitor.accept(resourceResolver.resolve(page));
            interimPaths.addAll(contentVisitor.getContentPaths());
            interimPaths.addAll(contentVisitor.getDamPaths());
            interimPaths.addAll(contentVisitor.getXfPaths());
        }
        return interimPaths;
    }

    private void iterateContentPaths(ResourceResolver resourceResolver, Set<String> contentPaths, Set<String> xfPaths,
                                     Set<String> damPaths, ContentVisitor contentVisitor, Set<String> pages) {
        for(String page : pages) {
            contentVisitor.accept(resourceResolver.resolve(page));
            contentPaths.addAll(contentVisitor.getContentPaths());
            xfPaths.addAll(contentVisitor.getXfPaths());
            damPaths.addAll(contentVisitor.getDamPaths());
        }
    }

    private String packagePaths(ResourceResolver resourceResolver, Set<String> allPaths, String packageName, String packageDescription)
            throws IOException, RepositoryException, PackageException {
        Map<String, String> packageDefinitionProperties = new HashMap<>();
        // ACL Handling
        packageDefinitionProperties.put(JcrPackageDefinition.PN_AC_HANDLING,
                AccessControlHandling.OVERWRITE.toString());

        // Package Description
        packageDefinitionProperties.put(JcrPackageDefinition.PN_DESCRIPTION, packageDescription);
        Set<@NotNull Resource> packageResources = allPaths.stream().map(resourceResolver::resolve).collect(Collectors.toSet());
        try(JcrPackage jcrPackage = packageHelper.createPackage(packageResources,
                resourceResolver.adaptTo(Session.class), DEFUALT_GROUP_NAME, packageName, DEFUALT_VERSION,
                PackageHelper.ConflictResolution.Replace, packageDefinitionProperties)){
            // Add thumbnail to the package definition
            packageHelper.addThumbnail(jcrPackage, resourceResolver.getResource(QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH));
            final JcrPackageManager packageManager = packaging.getPackageManager(resourceResolver.adaptTo(Session.class));
            packageManager.assemble(jcrPackage, null);
            return jcrPackage.getNode().getPath();
        }
    }
}

Add the following Content Visitor class to iterate resources as shown below:

package com.aem.operations.core.visitors;

import java.util.Arrays;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
import java.util.regex.Pattern;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.resource.AbstractResourceVisitor;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceUtil;
import org.apache.sling.api.resource.ValueMap;
import org.jetbrains.annotations.NotNull;

public class ContentVisitor extends AbstractResourceVisitor {

    private static final String CONTENT_SLASH = "/content/";
    private static final String CONTENT_DAM_SLASH = "/content/dam/";
    private static final String CONTENT_XF_SLASH = "/content/experience-fragments/";

    Pattern htmlPattern = Pattern.compile(".*\\<[^>]+>.*", Pattern.DOTALL);

    Set<String> contentPaths = new HashSet<>();
    Set<String> xfPaths = new HashSet<>();
    Set<String> damPaths = new HashSet<>();

    @Override
    public final void accept(final Resource resource) {
        if (null != resource && !ResourceUtil.isNonExistingResource(resource)) {
            final ValueMap properties = resource.adaptTo(ValueMap.class);
            final String primaryType = properties.get(ResourceResolver.PROPERTY_RESOURCE_TYPE, StringUtils.EMPTY);
            if(StringUtils.isNoneEmpty(primaryType) || StringUtils.startsWith(resource.getPath(), "/content/dam/content-fragments")){
                visit(resource);
            }
            this.traverseChildren(resource.listChildren());
        }
    }

    @Override
    protected void traverseChildren(final @NotNull Iterator<Resource> children) {
        while (children.hasNext()) {
            final Resource child = children.next();
            accept(child);
        }
    }

    @Override
    protected void visit(final @NotNull Resource resource) {
        resource.getValueMap().entrySet().forEach(property -> {
            Object prop = property.getValue();
            if (prop.getClass() == String[].class) {
                List<String> propertyValue = Arrays.asList((String[]) prop);
                if (!propertyValue.isEmpty()) {
                    propertyValue.stream().filter(s -> StringUtils.isNotEmpty(s) && StringUtils.startsWith(s, CONTENT_SLASH) && !htmlPattern.matcher(s).matches()).forEach(this::populateValue);
                }
            } else if (prop.getClass() == String.class) {
                String propertyValue = (String) prop;
                if (StringUtils.isNotEmpty(propertyValue) && StringUtils.startsWith(propertyValue, CONTENT_SLASH) && !htmlPattern.matcher(propertyValue).matches()) {
                    populateValue(propertyValue);
                }
            }
        });
    }

    private void populateValue(String value) {
        if(StringUtils.startsWith(value, CONTENT_DAM_SLASH)) {
            damPaths.add(value);
        } else if(StringUtils.startsWith(value, CONTENT_XF_SLASH)) {
            xfPaths.add(value+"/jcr:content");
        } else {
            contentPaths.add(StringUtils.substringBeforeLast(value, ".html")+"/jcr:content");
        }
    }

    public Set<String> getContentPaths() {
        return contentPaths;
    }

    public Set<String> getXfPaths() {
        return xfPaths;
    }

    public Set<String> getDamPaths() {
        return damPaths;
    }
}

Add the following Report Retry Utils as shown below:

package com.aem.operations.core.utils;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class ReportRetryUtils {
    private static final Logger log = LoggerFactory.getLogger(ReportRetryUtils.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int i = 0; i < maxTimes; i++) {
            try {
                call.process();
                if(i == (maxTimes-1)) {
                    return true;
                }
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (i + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

Add the following HTL to render the page:

<!DOCTYPE html>
<head>
   <link rel="shortcut icon" href="/libs/granite/core/content/login/favicon.ico">
</head>
<body class="coral--light foundation-layout-util-maximized-alt">
   <sly data-sly-call="${clientLib.all @ categories=['coralui3','granite.ui.coral.foundation','granite.ui.shell','cq.authoring.dialog', 'perfectpackage.base']}"
      data-sly-use.clientLib="/libs/granite/sightly/templates/clientlib.html"/>
   <coral-shell-header aria-hidden="false" aria-label="Header Bar"
      class="coral--dark granite-shell-header coral3-Shell-header"
      role="region">
      <coral-shell-header-home aria-level="2" class="globalnav-toggle" data-globalnav-toggle-href="/" role="heading">
         <a class="coral3-Shell-homeAnchor" href="/"
            icon="adobeExperienceManagerColor" is="coral-shell-homeanchor"
            style="display: inline-block; padding-right: 0;">
            <coral-icon
               aria-label="adobe experience manager color"
               class="coral3-Icon coral3-Shell-homeAnchor-icon coral3-Icon--adobeExperienceManagerColor coral3-Icon--sizeM"
               icon="adobeExperienceManagerColor" role="img"
               size="M"></coral-icon>
            <coral-shell-homeanchor-label>Adobe Experience Manager
            </coral-shell-homeanchor-label>
         </a>
         <span style="line-height: 2.375rem;">/ AEM Operations / ${properties.jcr:title} </span>
      </coral-shell-header-home>
   </coral-shell-header>
   <div class="foundation-layout-panel">
      <div class="foundation-layout-panel-header">
         <betty-titlebar>
            <betty-titlebar-title>
               <span aria-level="1" class="granite-title" role="heading">${properties.jcr:title}</span>
            </betty-titlebar-title>
            <betty-titlebar-primary></betty-titlebar-primary>
            <betty-titlebar-secondary>
            </betty-titlebar-secondary>
         </betty-titlebar>
      </div>
      <div class="content-container-inner coral-Well" style="width:60%; margin:0 auto;">
         <div class="acsCommons-System-Notifications-Form-row">
		   <label class="coral-Form-fieldlabel">List of Paths</label>
		   <label> Note: Add the list of paths has a Line feed (No need for comma separation)</label>
		   <textarea
		      class="coral-Textfield coral-Textfield--multiline acsCommons-System-Notifications-Page-input--textarea"
		      rows="6"
		      name="./pathsList"></textarea>
		</div>
         <sly data-sly-use.packprops="com.aem.operations.core.models.PerfectPackageModel">
            <sly data-sly-resource="${packprops.formResource}"/>
         </sly>
         <coral-actionbar-item class="coral3-ActionBar-item">
            <button class="coral3-Button coral3-Button--primary package-initiator" icon="checkCircle" iconsize="S"
               is="coral-button"
               size="M" variant="primary">
               <coral-icon alt="" class="coral3-Icon coral3-Icon--sizeS coral3-Icon--checkCircle" icon="folderGear"
                  size="S"></coral-icon>
               <coral-button-label>
                  Create Content Backup
               </coral-button-label>
            </button>
         </coral-actionbar-item>
          <!-- The Modal -->
          <div id="popup-modal" class="modal">
              <!-- Modal content -->
              <div class="modal-content">
                  <span class="close">&times;</span>
                  <span class="loading">Loading</span>
                  <span class="loader"></span>
              </div>
          </div>
      </div>
   </div>
</body>

Add the following JS code to process the ajax request as shown below:

var Coral = window.Coral || {},
    Granite = window.Granite || {};

(function (window, document, $, Coral) {
    "use strict";
    $(document).on("foundation-contentloaded", function (e) {

        var SITE_PATH = "/conf/aemoperations/settings/tools/perfectpackage-initiator.html",
            ui = $(window).adaptTo("foundation-ui");

        if (window.location.href.indexOf(SITE_PATH) < 0) {
            return;
        }

        $(document).off("change", ".close").on("change", ".close", function (event) {
            $(".modal").hide();
        });

        function getValueByName(fieldName, isMandatory) {
            var fieldValue = ($("input[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("input[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("input[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getValueByNameArea(fieldName, isMandatory) {
            var fieldValue = ($("textarea[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("textarea[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("textarea[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        $(document).off("click", ".package-initiator").on("click", ".package-initiator", function (event) {
            event.preventDefault();

            var packageName = getValueByName('./packageName', true),
                packageDescription = getValueByName('./packageDescription', true),
                pathsList = getValueByNameArea('./pathsList', true);

            var formData = new FormData();
            formData.append("packageName", packageName);
            formData.append("packageDescription", packageDescription);
            formData.append("pathsList", pathsList);

			$(".loading").html("PLEASE WAIT CREATINGING PACKAGE");
            $(".modal").show();

            $.ajax({
                url: "/bin/triggerPerfectPackage",
                method: "POST",
                async: true,
                cache: false,
                contentType: false,
                processData: false,
                data: formData
            }).done(function (data) {
                if (data && data.message){
                    ui.notify("Success", data.message, "success");
                    var dialog = new Coral.Dialog();
                    dialog.id = 'dialogSuccess';
                    dialog.header.innerHTML = 'Success';
                    dialog.content.innerHTML = data.message;
                    dialog.footer.innerHTML = '<button class="ok-button" is="coral-button" variant="primary" icon="check" coral-close>OK</button>';
                    dialog.variant = 'success';
                    dialog.closable = "on";
                    dialog.show();
                    $(".modal").hide();
                }else{
                    ui.notify("Error", "Unable to create package", "error");
                }
            }).fail(function (data) {
				$(".modal").hide();
                if (data && data.responseJSON && data.responseJSON.message){
                    ui.notify("Error", data.responseJSON.message, "error");
                }else{
                    ui.notify("Error", "Unable to create package", "error");
                }
            });
        });
    });
})(window, document, $, Coral);

Once the code is built and deployed you will be able to access the tool by navigating the:

Tools -> {Tool Section} -> {Tool Name}

Now you can input the page paths as a line feed (no need to separate the paths using commas or any characters)

Provide the desired package name and description and you can see the package path will be prompted and you will be able to download the built package by clicking on the link:

Perfect package built and has all the references

Working Code:
You can also access the working code on my GitHub repository link: https://github.com/kiransg89/aemoperations

Note: You can access this package over domain on AEM / AEMaaCS and over any environments and architects can handle the permissions by adding appropriate rep:policy on cq generator node or conf page

AEM Tool – Create a package using a list of paths

Problem statement:

How to create a package using a list of paths?

Avoid adding individual paths manually in the CRX Package manager

Option to package current page or pull children

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

AEM operations
Managing templates
Cloud configurations
ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Package Creator is an extension of the ACS Commons Package handler, but it’s UI friendly and easy to access. You can input the paths to be packaged as a line feed and no need to separate them by a comma. Once the package is ready it will be built and prompted with a download option.

In order to create a package please use the Tool generator to give your tool name and descriptions: https://kiransg.com/2022/11/24/aem-tool-create-generate-tool-from-scratch

Add the following Sling model fields as shown below to accept the inputs:

package com.aem.operations.core.models;

import javax.inject.Named;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.GeneratedDialog;
import com.adobe.acs.commons.mcp.form.SelectComponent;
import lombok.Getter;
import lombok.Setter;

@Model(adaptables = { Resource.class,SlingHttpServletRequest.class }, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class PackageCreatorModel extends GeneratedDialog {

    public enum GetChildren {
        NO, YES
    }

    @Getter
    @Setter
    @Named(value = "packageName")
    @FormField(
            name = "Package Name",
            category = "General",
            required = true,
            description = "Enter the package name",
            hint = "Name")
    private String packageName;

    @Getter
    @Setter
    @Named(value = "packageDescription")
    @FormField(
            name = "Package Description",
            category = "General",
            required = true,
            description = "Enter the package Description",
            hint = "Description")
    private String packageDescription;

    @Getter
    @Setter
    @Named(value = "getChildren")
    @FormField(
            name = "Get Children",
            description = "child pages will be pulled",
            required = true,
            component = SelectComponent.EnumerationSelector.class,
            options = {"horizontal", "default=NO"})
    GetChildren getChildren = GetChildren.NO;
}

Add the following Servlet to process the request as shown below:

package com.aem.operations.core.servlets;

import com.adobe.acs.commons.packaging.PackageHelper;
import com.adobe.granite.rest.Constants;
import com.aem.operations.core.models.PackageCreatorModel;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.vault.fs.io.AccessControlHandling;
import org.apache.jackrabbit.vault.packaging.*;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.sling.servlets.post.JSONResponse;
import org.jetbrains.annotations.Nullable;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.*;
import java.util.stream.Collectors;

@Component(
        service = { Servlet.class },
        property = {
                "sling.servlet.paths=" + PackageCreatorServlet.RESOURCE_PATH,
                "sling.servlet.methods=POST"
        }
)
public class PackageCreatorServlet extends SlingAllMethodsServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger LOGGER = LoggerFactory.getLogger(PackageCreatorServlet.class);
    public static final String RESOURCE_PATH = "/bin/triggerPackageCreator";
    private static final String MESSAGE = "message";
    private static final String CONTENT_DAM_SLASH = "/content/dam/";

    private static final String DEFUALT_GROUP_NAME = "my_packages";
    private static final String DEFUALT_VERSION = "1.0";
    private static final String QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH = "/apps/acs-commons/components/utilities/packager/query-packager/definition/package-thumbnail.png";

    @Reference
    private PackageHelper packageHelper;

    @Reference
    private Packaging packaging;

    @Override
    protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response) throws ServletException, IOException {
        // Set response headers.
        response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE);
        response.setCharacterEncoding(Constants.DEFAULT_CHARSET);

        JsonObject jsonResponse = new JsonObject();
        ResourceResolver resourceResolver = request.getResourceResolver();

        String packageName = request.getParameter("packageName");
        String packageDescription = request.getParameter("packageGroup");
        String pathsList = request.getParameter("pathsList");
        String getChildren = request.getParameter("getChildren");

        String packagePath = StringUtils.EMPTY;
        if(StringUtils.isNotEmpty(pathsList)) {
            packagePath = createPackage(resourceResolver, packageName, packageDescription, pathsList, getChildren);
        }

        if(StringUtils.isNotEmpty(packagePath)) {
            jsonResponse.addProperty(MESSAGE, "Package creation is Successful and path is <a href=\"" + packagePath + "\">" + packagePath + "</a>");
        } else {
            LOGGER.error("Unable to process package creation");
        }
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }

    private String createPackage(ResourceResolver resourceResolver, String packageName, String packageDescription,
                                 String pathsList, String getChildren) throws IOException {
        Set<String> pages = new HashSet<>(Arrays.asList(StringUtils.split(pathsList, '\n'))).stream().map(String::trim).collect(Collectors.toSet());
        Set<Resource> packageResources = pages.stream().map(path -> getResource(resourceResolver, path, getChildren)).collect(Collectors.toSet());
        if(null != packageResources && !packageResources.isEmpty()) {
            Map<String, String> packageDefinitionProperties = new HashMap<>();
            // ACL Handling
            packageDefinitionProperties.put(JcrPackageDefinition.PN_AC_HANDLING,
                    AccessControlHandling.OVERWRITE.toString());
            // Package Description
            packageDefinitionProperties.put(JcrPackageDefinition.PN_DESCRIPTION, packageDescription);
            try(JcrPackage jcrPackage = packageHelper.createPackage(packageResources,
                    resourceResolver.adaptTo(Session.class), DEFUALT_GROUP_NAME, packageName, DEFUALT_VERSION,
                    PackageHelper.ConflictResolution.Replace, packageDefinitionProperties)){
                // Add thumbnail to the package definition
                packageHelper.addThumbnail(jcrPackage, resourceResolver.getResource(QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH));
                final JcrPackageManager packageManager = packaging.getPackageManager(resourceResolver.adaptTo(Session.class));
                packageManager.assemble(jcrPackage, null);
                return jcrPackage.getNode().getPath();
            } catch (RepositoryException | PackageException e) {
                LOGGER.error("Exception occurred during package creation {}", e.getMessage());
            }
        }
        return StringUtils.EMPTY;
    }

    private @Nullable Resource getResource(ResourceResolver resourceResolver, String path, String getChildren) {
        if(StringUtils.equalsIgnoreCase(PackageCreatorModel.GetChildren.NO.toString(), getChildren) && !StringUtils.contains(path, CONTENT_DAM_SLASH)) {
            return resourceResolver.resolve(path + "/jcr:content");
        }
        return resourceResolver.resolve(path);
    }
}

Add the following HTL to render the page:

<!DOCTYPE html>
<head>
   <link rel="shortcut icon" href="/libs/granite/core/content/login/favicon.ico">
</head>
<body class="coral--light foundation-layout-util-maximized-alt">
   <sly data-sly-call="${clientLib.all @ categories=['coralui3','granite.ui.coral.foundation','granite.ui.shell','cq.authoring.dialog', 'packagecreator.base']}"
      data-sly-use.clientLib="/libs/granite/sightly/templates/clientlib.html"/>
   <coral-shell-header aria-hidden="false" aria-label="Header Bar"
      class="coral--dark granite-shell-header coral3-Shell-header"
      role="region">
      <coral-shell-header-home aria-level="2" class="globalnav-toggle" data-globalnav-toggle-href="/" role="heading">
         <a class="coral3-Shell-homeAnchor" href="/"
            icon="adobeExperienceManagerColor" is="coral-shell-homeanchor"
            style="display: inline-block; padding-right: 0;">
            <coral-icon
               aria-label="adobe experience manager color"
               class="coral3-Icon coral3-Shell-homeAnchor-icon coral3-Icon--adobeExperienceManagerColor coral3-Icon--sizeM"
               icon="adobeExperienceManagerColor" role="img"
               size="M"></coral-icon>
            <coral-shell-homeanchor-label>Adobe Experience Manager
            </coral-shell-homeanchor-label>
         </a>
         <span style="line-height: 2.375rem;">/ AEM Operations / ${properties.jcr:title} </span>
      </coral-shell-header-home>
   </coral-shell-header>
   <div class="foundation-layout-panel">
      <div class="foundation-layout-panel-header">
         <betty-titlebar>
            <betty-titlebar-title>
               <span aria-level="1" class="granite-title" role="heading">${properties.jcr:title}</span>
            </betty-titlebar-title>
            <betty-titlebar-primary></betty-titlebar-primary>
            <betty-titlebar-secondary>
            </betty-titlebar-secondary>
         </betty-titlebar>
      </div>
      <div class="content-container-inner coral-Well" style="width:60%; margin:0 auto;">
         <div class="acsCommons-System-Notifications-Form-row">
		   <label class="coral-Form-fieldlabel">List of Paths</label>
		   <label> Note: Add the list of paths has a Line feed (No need for comma separation)</label>
		   <textarea
		      class="coral-Textfield coral-Textfield--multiline acsCommons-System-Notifications-Page-input--textarea"
		      rows="6"
		      name="./pathsList"></textarea>
		 </div>
         <sly data-sly-use.packprops="com.aem.operations.core.models.PackageCreatorModel">
            <sly data-sly-resource="${packprops.formResource}"/>
         </sly>
         <coral-actionbar-item class="coral3-ActionBar-item">
            <button class="coral3-Button coral3-Button--primary package-initiator" icon="checkCircle" iconsize="S"
               is="coral-button"
               size="M" variant="primary">
               <coral-icon alt="" class="coral3-Icon coral3-Icon--sizeS coral3-Icon--checkCircle" icon="actions"
                  size="S"></coral-icon>
               <coral-button-label>
                  Create Package
               </coral-button-label>
            </button>
         </coral-actionbar-item>
          <!-- The Modal -->
          <div id="popup-modal" class="modal">
              <!-- Modal content -->
              <div class="modal-content">
                  <span class="close">&times;</span>
                  <span class="loading">Loading</span>
                  <span class="loader"></span>
              </div>
          </div>
      </div>
   </div>
</body>

Add the following JS code to process the ajax request as shown below:

var Coral = window.Coral || {},
    Granite = window.Granite || {};

(function (window, document, $, Coral) {
    "use strict";
    $(document).on("foundation-contentloaded", function (e) {

        var SITE_PATH = "/conf/aemoperations/settings/tools/create-package-initiator.html",
            ui = $(window).adaptTo("foundation-ui");

        if (window.location.href.indexOf(SITE_PATH) < 0) {
            return;
        }

        $(document).off("change", ".close").on("change", ".close", function (event) {
            $(".modal").hide();
        });

        function getValueByName(fieldName, isMandatory) {
            var fieldValue = ($("input[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("input[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("input[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getValueByNameArea(fieldName, isMandatory) {
            var fieldValue = ($("textarea[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("textarea[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("textarea[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        $(document).off("click", ".package-initiator").on("click", ".package-initiator", function (event) {
            event.preventDefault();

            var packageName = getValueByName('./packageName', true),
                packageDescription = getValueByName('./packageDescription', true),
                pathsList = getValueByNameArea('./pathsList', true),
                getChildren = getValueByName('./getChildren', true);

            var formData = new FormData();
            formData.append("packageName", packageName);
            formData.append("packageDescription", packageDescription);
            formData.append("pathsList", pathsList);
            formData.append("getChildren", getChildren);

			$(".loading").html("PLEASE WAIT CREATINGING PACKAGE");
            $(".modal").show();

            $.ajax({
                url: "/bin/triggerPackageCreator",
                method: "POST",
                async: true,
                cache: false,
                contentType: false,
                processData: false,
                data: formData
            }).done(function (data) {
                if (data && data.message){
                    ui.notify("Success", data.message, "success");
                    var dialog = new Coral.Dialog();
                    dialog.id = 'dialogSuccess';
                    dialog.header.innerHTML = 'Success';
                    dialog.content.innerHTML = data.message;
                    dialog.footer.innerHTML = '<button class="ok-button" is="coral-button" variant="primary" icon="check" coral-close>OK</button>';
                    dialog.variant = 'success';
                    dialog.closable = "on";
                    dialog.show();
                    $(".modal").hide();
                }else{
                    ui.notify("Error", "Unable to create package ", "error");
                }
            }).fail(function (data) {
				$(".modal").hide();
                if (data && data.responseJSON && data.responseJSON.message){
                    ui.notify("Error", data.responseJSON.message, "error");
                }else{
                    ui.notify("Error", "Unable to create package", "error");
                }
            });
        });
    });
})(window, document, $, Coral);

Once the code is built and deployed you will be able to access the tool by navigating the:

Tools -> {Tool Section} -> {Tool Name}

Now you can input the page paths as a line feed (no need to separate the paths using commas or any characters)

Provide the list paths to be backed up, desired package name, and description and select the dropdown to pull children or not. Once the package is built you can see the package path will be prompted and you will be able to download the package by clicking on the built package link:

The package is built with the exact page location

Working Code:
You can also access the working code on my GitHub repository link: https://github.com/kiransg89/aemoperations

Note: You can access this package over domain on AEM / AEMaaCS and over any environments and architects can handle the permissions by adding appropriate rep:policy on cq generator node or conf page

AEM Tool – Create / Generate tool from scratch

Problem Statement:

Create a tool similar to AEM Operations or ACS Commons for easy access and to run any maintenance process/tasks.

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

AEM operations
Managing templates
Cloud configurations
ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

In order to create any tool from scratch takes a lot of time and man hours and once all the configs are ready then it takes more time to develop the services and servlet to handle the business logic.

By using this tool, you can avoid all kinds of configurations and initial setup and kick start with your own first tool from scratch

The tool generator consists of the:

Sling model – to generate fields and handle inputs
Servlet – to process request
Component – to handle view (HTL), CSS, and JS

All the above sample scratch setups along with ready to check-in code will be added to the code base.

How to use the tool?

1. Click on the link to download the tool generator package and install it into your local instance

toolgenerator.all-1.0.0-SNAPSHOT Download

2. Once the package is installed go to sites -> tools -> and select the Tool Generator section

3. Select Tool Settings on the top right hand corner and provide your local repository paths like sling model path, servlet path, apps path, conf path, and CQ path if it already exists

4. Once all the settings are authored save the settings

5. After coming to the generator page provide your tool name and tool description

6. If you have an existing tool then select Yes else Select No and provide the Tool Section name

7. Click on Create tool to create your new tool from scratch

Check your repository for all the file changes as shown below:

You can also check your new tool component and other configs on CRXDE

You can also visit your new tool by accessing the tool section (sites -> Tools -> {Your section name})

For more info on how to add a Sling model field please visit the below link:

Once your Sling model, servlet, and other things are ready make sure you add the following filter in your META-INF folder and keep it merge as shown below:

This would avoid replacing/overriding any other tools like ACS Commons or your other repo tools.