AEM Tool – Publish / UnPublish / Delete List of pages

Problem statement:

Can we Publish / Un Publish / Delete the list of paths mentioned in an excel sheet? Or provided has a linefeed

Introduction:

AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations

For example:

  1. AEM operations
  2. Managing templates
  3. Cloud configurations
  4. ACS Commons tools etc..

Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.

Usually, product owners or authors would like to Publish certain pages like offer/product/article pages based on business requirements and also would like to unpublish and delete to clean up the pages which are unnecessary.

This process will be really helpful during excel sheet based migration.

The list Replication process usually validates the paths and if it exists then activates or deactivates or deletes pages or content paths. It accepts an excel sheet or list of paths as a line feed.

You can also provide a list of agents to be activated/deactivated to for example only to Brightspot or Brightcove connector.

In order to create a tool please use the Tool generator to give your tool name and description: https://kiransg.com/2022/11/24/aem-tool-create-generate-tool-from-scratch

Add the following Sling model fields as shown below to accept the inputs:

package com.aem.operations.core.models;

import com.adobe.acs.commons.mcp.form.*;
import lombok.Getter;
import lombok.Setter;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import javax.inject.Named;
import java.io.InputStream;

@Model(adaptables = { Resource.class,SlingHttpServletRequest.class }, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class ListTreeActivationModel extends GeneratedDialog {

    public enum QueueMethod {
        USE_MCP_QUEUE, USE_PUBLISH_QUEUE, MCP_AFTER_10K
    }

    public enum ReplicationAction {
        PUBLISH, UNPUBLISH, DELETE
    }

    @Getter
    @Setter
    @Named(value = "repPathExcel")
    @FormField(name = "Replication Excel", description = "Upload Replication Excel", component = FileUploadComponent.class)
    private InputStream repPathExcel = null;

    @Getter
    @Setter
    @Named(value = "queueMethod")
    @FormField(
            name = "Queueing Method",
            description = "For small publishing tasks, standard is sufficient.  For large folder trees, MCP is recommended.",
            required = true,
            component = SelectComponent.EnumerationSelector.class,
            options = {"horizontal", "default=USE_MCP_QUEUE"})
    QueueMethod queueMethod = QueueMethod.USE_MCP_QUEUE;

    @Getter
    @Setter
    @Named(value = "agents")
    @FormField(name = "Agents",
            component = TextfieldComponent.class,
            hint = "(leave blank for default agents)",
            description = "Publish agents to use, if blank then all default agents will be used. Multiple agents can be listed using commas or regex.")
    private String agents;

    @Getter
    @Setter
    @Named(value = "reAction")
    @FormField(name = "Action",
            component = SelectComponent.EnumerationSelector.class,
            description = "Publish or Unpublish?",
            options = "default=PUBLISH")
    ReplicationAction reAction = ReplicationAction.PUBLISH;
}

Add the following Servlet to process the request as shown below:

package com.aem.operations.core.servlets;

import java.io.IOException;
import java.io.PrintWriter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
import javax.jcr.Session;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import com.aem.operations.core.models.ListTreeActivationModel;
import com.aem.operations.core.utils.RetryUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.servlets.SlingAllMethodsServlet;
import org.apache.sling.servlets.post.JSONResponse;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.acs.commons.data.CompositeVariant;
import com.adobe.acs.commons.data.Spreadsheet;
import com.adobe.granite.rest.Constants;
import com.day.cq.replication.AgentFilter;
import com.day.cq.replication.ReplicationActionType;
import com.day.cq.replication.ReplicationException;
import com.day.cq.replication.ReplicationOptions;
import com.day.cq.replication.Replicator;
import com.google.gson.Gson;
import com.google.gson.JsonObject;

@Component(service = { Servlet.class }, property = { "sling.servlet.paths=" + ListActiavationServlet.RESOURCE_PATH,
        "sling.servlet.methods=POST" })
public class ListActiavationServlet extends SlingAllMethodsServlet {

    private static final long serialVersionUID = 1L;
    private static final Logger LOGGER = LoggerFactory.getLogger(ListActiavationServlet.class);
    public static final String RESOURCE_PATH = "/bin/triggerListActivation";
    private static final String MESSAGE = "message";
    private static final String DESTINATION_PATH = "destination";
    private static int ASYNC_LIMIT = 10000;


    private List<String> agentList = new ArrayList<>();
    private Spreadsheet spreadsheet;
    AtomicInteger replicationCount = new AtomicInteger();

    @Reference
    Replicator replicator;

    @Override
    protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response)
            throws ServletException, IOException {

        // Set response headers.
        response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE);
        response.setCharacterEncoding(Constants.DEFAULT_CHARSET);

        JsonObject jsonResponse = new JsonObject();

        String queueMethod = request.getParameter("queueMethod");
        String agents = request.getParameter("agents");
        String reAction = request.getParameter("reAction");
        String pathsList = request.getParameter("pathsList");

        @Nullable
        RequestParameter inputPackStream = request.getRequestParameter("file");
        if(inputPackStream != null) {
            spreadsheet = new Spreadsheet(inputPackStream, DESTINATION_PATH).buildSpreadsheet();
        }
        if (null != spreadsheet) {
            AgentFilter replicationAgentFilter = prepareAgents(agents);
            if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.PUBLISH.toString(), reAction)) {
                activateTreeStructure(request.getResourceResolver(), queueMethod, reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Activated all the paths provided in Excel sheet");
            } else if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.UNPUBLISH.toString(), reAction)) {
                deactivateTreeStructure(request.getResourceResolver(), reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deactivated all the paths provided in Excel sheet");
            } else {
                deleteTree(request.getResourceResolver());
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deleted all the paths provided in Excel sheet");
            }
        } else if(StringUtils.isNotEmpty(pathsList)){
            AgentFilter replicationAgentFilter = prepareAgents(agents);
            Set<String> pages = new HashSet<>(Arrays.asList(StringUtils.split(pathsList, '\n'))).stream().map(String::trim).collect(Collectors.toSet());
            if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.PUBLISH.toString(), reAction)) {
                activateTreeStructure(request.getResourceResolver(), pages, queueMethod, reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Activated all the paths provided in Excel sheet");
            } else if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.UNPUBLISH.toString(), reAction)) {
                deactivateTreeStructure(request.getResourceResolver(), pages, reAction, replicationAgentFilter);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deactivated all the paths provided in Line Feed");
            } else {
                deleteTree(request.getResourceResolver(), pages);
                jsonResponse.addProperty(MESSAGE, "Succcessfully Deleted all the paths provided in Line Feed");
            }
        } else {
            jsonResponse.addProperty(MESSAGE, "Unable to process Activation request because of invalid Inputs");
        }
        try (PrintWriter out = response.getWriter()) {
            out.print(new Gson().toJson(jsonResponse));
        }
    }

    private AgentFilter prepareAgents(String agents) {
        AgentFilter replicationAgentFilter;
        if (StringUtils.isEmpty(agents)) {
            replicationAgentFilter = AgentFilter.DEFAULT;
        } else {
            agentList = Arrays.asList(agents.toLowerCase(Locale.ENGLISH).split(","));
            replicationAgentFilter = agent -> agentList.stream()
                    .anyMatch(p -> p.matches(agent.getId().toLowerCase(Locale.ENGLISH)));
        }
        return replicationAgentFilter;
    }

    private void activateTreeStructure(@NotNull ResourceResolver resourceResolver, String queueMethod, String reAction, AgentFilter replicationAgentFilter) {
        spreadsheet.getDataRowsAsCompositeVariants()
                .forEach(row -> performReplication(resourceResolver, getString(row, DESTINATION_PATH), queueMethod, reAction, replicationAgentFilter));
    }

    private void deactivateTreeStructure(@NotNull ResourceResolver resourceResolver, String reAction, AgentFilter replicationAgentFilter) {
        spreadsheet.getDataRowsAsCompositeVariants()
                .forEach(row -> performAsynchronousReplication(resourceResolver, getString(row, DESTINATION_PATH), reAction, replicationAgentFilter));
    }

    private void activateTreeStructure(@NotNull ResourceResolver resourceResolver, Set<String> pages, String queueMethod, String reAction, AgentFilter replicationAgentFilter) {
        pages.stream().forEach(path -> performReplication(resourceResolver, path, queueMethod, reAction, replicationAgentFilter));
    }

    private void deactivateTreeStructure(@NotNull ResourceResolver resourceResolver, Set<String> pages, String reAction, AgentFilter replicationAgentFilter) {
        pages.stream().forEach(path -> performAsynchronousReplication(resourceResolver, path, reAction, replicationAgentFilter));
    }

    private void performReplication(@NotNull ResourceResolver resourceResolver, String path, String queueMethod, String reAction, AgentFilter replicationAgentFilter) {
        int counter = replicationCount.incrementAndGet();
        if (StringUtils.equalsIgnoreCase(ListTreeActivationModel.QueueMethod.USE_MCP_QUEUE.toString(), queueMethod)
                || (StringUtils.equalsIgnoreCase(ListTreeActivationModel.QueueMethod.MCP_AFTER_10K.toString(),
                queueMethod) && counter >= ASYNC_LIMIT)) {
            performSynchronousReplication(resourceResolver, path, reAction, replicationAgentFilter);
        } else {
            performAsynchronousReplication(resourceResolver, path, reAction, replicationAgentFilter);
        }
    }

    private void performSynchronousReplication(@NotNull ResourceResolver resourceResolver, String path, String reAction, AgentFilter replicationAgentFilter) {
        ReplicationOptions options = buildOptions(replicationAgentFilter);
        options.setSynchronous(true);
        scheduleReplication(resourceResolver, options, path, reAction);
    }

    private void performAsynchronousReplication(@NotNull ResourceResolver resourceResolver, String path, String reAction, AgentFilter replicationAgentFilter) {
        ReplicationOptions options = buildOptions(replicationAgentFilter);
        options.setSynchronous(false);
        scheduleReplication(resourceResolver, options, path, reAction);
    }

    private void scheduleReplication(@NotNull ResourceResolver resourceResolver, ReplicationOptions options,
                                     String path, String reAction) {
        Session session = resourceResolver.adaptTo(Session.class);
        boolean isPublish = StringUtils
                .equalsIgnoreCase(ListTreeActivationModel.ReplicationAction.PUBLISH.toString(), reAction);
        try {
            replicator.replicate(session,
                    isPublish ? ReplicationActionType.ACTIVATE : ReplicationActionType.DEACTIVATE, path, options);
        } catch (ReplicationException e) {
            LOGGER.error("Error occured during replication {}", e.getMessage());
        }
    }

    @SuppressWarnings("rawtypes")
    private void deleteTree(ResourceResolver resourceResolver) {
        try {
            RetryUtils.withRetry(30, 500, () -> {
                Iterator<Map<String, CompositeVariant>> spredIterator = spreadsheet.getDataRowsAsCompositeVariants()
                        .iterator();
                while (spredIterator.hasNext()) {
                    Map<String, CompositeVariant> row = spredIterator.next();
                    deleteContent(resourceResolver, getString(row, DESTINATION_PATH));
                }
                commitChanges(resourceResolver);
            });
        } catch (Exception e) {
            LOGGER.error("Error occured during Retrying {}", e.getMessage());
        }
    }

    private void deleteTree(ResourceResolver resourceResolver, Set<String> pages) {
        try {
            RetryUtils.withRetry(30, 500, () -> {
                pages.forEach(path -> deleteContent(resourceResolver, path));
                commitChanges(resourceResolver);
            });
        } catch (Exception e) {
            LOGGER.error("Error occured during Retrying {}", e.getMessage());
        }
    }

    private void deleteContent(ResourceResolver resourceResolver, String destinationPath) {
        try {
            @Nullable
            Resource destinationResource = resourceResolver.getResource(destinationPath);
            if (null != destinationResource) {
                resourceResolver.delete(destinationResource);
            }
        } catch (PersistenceException e) {
            LOGGER.error("unable to delete content {}", e.getMessage());
        }
    }

    private void commitChanges(ResourceResolver resourceResolver) throws PersistenceException {
        if (resourceResolver.hasChanges()) {
            resourceResolver.commit();
            resourceResolver.refresh();
        }
    }

    private ReplicationOptions buildOptions(AgentFilter replicationAgentFilter) {
        ReplicationOptions options = new ReplicationOptions();
        options.setFilter(replicationAgentFilter);
        return options;
    }

    @SuppressWarnings({ "rawtypes", "unchecked" })
    private String getString(Map<String, CompositeVariant> row, String path) {
        CompositeVariant v = row.get(path.toLowerCase(Locale.ENGLISH));
        if (v != null) {
            return (String) v.getValueAs(String.class);
        } else {
            return null;
        }
    }
}

Add the following Retry Utils as shown below:

package com.aem.operations.core.utils;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryUtils {
    private static final Logger log = LoggerFactory.getLogger(RetryUtils.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int i = 0; i < maxTimes; i++) {
            try {
                call.process();
                return true;
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (i + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

Add the following HTL to render the page:

<!DOCTYPE html>
<head>
   <link rel="shortcut icon" href="/libs/granite/core/content/login/favicon.ico">
</head>
<body class="coral--light foundation-layout-util-maximized-alt">
   <sly data-sly-call="${clientLib.all @ categories=['coralui3','granite.ui.coral.foundation','granite.ui.shell','cq.authoring.dialog', 'listreplication.base']}"
      data-sly-use.clientLib="/libs/granite/sightly/templates/clientlib.html"/>
   <coral-shell-header aria-hidden="false" aria-label="Header Bar"
      class="coral--dark granite-shell-header coral3-Shell-header"
      role="region">
      <coral-shell-header-home aria-level="2" class="globalnav-toggle" data-globalnav-toggle-href="/" role="heading">
         <a class="coral3-Shell-homeAnchor" href="/"
            icon="adobeExperienceManagerColor" is="coral-shell-homeanchor"
            style="display: inline-block; padding-right: 0;">
            <coral-icon
               aria-label="adobe experience manager color"
               class="coral3-Icon coral3-Shell-homeAnchor-icon coral3-Icon--adobeExperienceManagerColor coral3-Icon--sizeM"
               icon="adobeExperienceManagerColor" role="img"
               size="M"></coral-icon>
            <coral-shell-homeanchor-label>Adobe Experience Manager
            </coral-shell-homeanchor-label>
         </a>
         <span style="line-height: 2.375rem;">/ AEM Operations / ${properties.jcr:title} </span>
      </coral-shell-header-home>
   </coral-shell-header>
   <div class="foundation-layout-panel">
      <div class="foundation-layout-panel-header">
         <betty-titlebar>
            <betty-titlebar-title>
               <span aria-level="1" class="granite-title" role="heading">${properties.jcr:title}</span>
            </betty-titlebar-title>
            <betty-titlebar-primary></betty-titlebar-primary>
            <betty-titlebar-secondary>
            </betty-titlebar-secondary>
         </betty-titlebar>
      </div>
      <div class="content-container-inner coral-Well" style="width:60%; margin:0 auto;">
         <div class="acsCommons-System-Notifications-Form-row">
		   <label class="coral-Form-fieldlabel">List of Paths</label>
		   <label> Note: Add the list of paths has a Line feed (No need for comma separation)</label>
		   <textarea
		      class="coral-Textfield coral-Textfield--multiline acsCommons-System-Notifications-Page-input--textarea"
		      rows="6"
		      name="./pathsList"></textarea>
		 </div>
		 <label> Note: Create a Excel Sheet with column called <b>destination</b> and the list of values and upload</label>
         <sly data-sly-use.packprops="com.aem.operations.core.models.ListTreeActivationModel">
            <sly data-sly-resource="${packprops.formResource}"/>
         </sly>
         <coral-actionbar-item class="coral3-ActionBar-item">
            <button class="coral3-Button coral3-Button--primary activation-initiator" icon="checkCircle" iconsize="S"
               is="coral-button"
               size="M" variant="primary">
               <coral-icon alt="" class="coral3-Icon coral3-Icon--sizeS coral3-Icon--checkCircle" icon="publish"
                  size="S"></coral-icon>
               <coral-button-label>
                  Start the Replication
               </coral-button-label>
            </button>
         </coral-actionbar-item>
          <!-- The Modal -->
          <div id="popup-modal" class="modal">
              <!-- Modal content -->
              <div class="modal-content">
                  <span class="close">&times;</span>
                  <span class="loading">Loading</span>
                  <span class="loader"></span>
              </div>
          </div>
      </div>
   </div>
</body>

Add the following JS code to process the ajax request as shown below:

var Coral = window.Coral || {},
    Granite = window.Granite || {};

(function (window, document, $, Coral) {
    "use strict";
    $(document).on("foundation-contentloaded", function (e) {

        var SITE_PATH = "/conf/aemoperations/settings/tools/listreplication-initiator.html",
            ui = $(window).adaptTo("foundation-ui");

        if (window.location.href.indexOf(SITE_PATH) < 0) {
            return;
        }

        $(document).off("change", ".close").on("change", ".close", function (event) {
            $(".modal").hide();
        });

        function getValueByName(fieldName, isMandatory) {
            var fieldValue = ($("input[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("input[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("input[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getValueByNameArea(fieldName, isMandatory) {
            var fieldValue = ($("textarea[name='" + fieldName + "']").val()).trim();
            if (!isMandatory) {
                return fieldValue;
            }
            if (!fieldValue || fieldValue.length === 0) {
                //for input fields
                $("textarea[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("textarea[name='" + fieldName + "']").attr('invalid', 'invalid');

                //for select fields
                $("coral-select[name='" + fieldName + "']").attr('aria-invalid', 'true');
                $("coral-select[name='" + fieldName + "']").attr('invalid', 'invalid');

                return;
            } else {
                return fieldValue;
            }
        }

        function getFileByName(fieldName) {
            var fieldInput = $("input[name='" + fieldName + "']");
            if(null != fieldInput && null != fieldInput[0]){
                return fieldInput[0].files[0];
            }
        }

        $(document).off("click", ".activation-initiator").on("click", ".activation-initiator", function (event) {
            event.preventDefault();

            var queueMethod = getValueByName('./queueMethod', false),
                agents = getValueByName('./agents', false),
                reAction = getValueByName('./reAction', false),
                pathsList = getValueByNameArea('./pathsList', true),
                repPathExcel = getFileByName('./repPathExcel');

            var formData = new FormData();
            if(null != repPathExcel){
                formData.append("file", repPathExcel, repPathExcel.name);
            }
            formData.append("queueMethod", queueMethod);
            formData.append("agents", agents);
            formData.append("reAction", reAction);
            formData.append("pathsList", pathsList);

			$(".loading").html("Replication in progress");
            $(".modal").show();

            $.ajax({
                url: "/bin/triggerListActivation",
                method: "POST",
                async: true,
                cache: false,
                contentType: false,
                processData: false,
                data: formData
            }).done(function (data) {
                if (data && data.message){
                    ui.notify("Success", data.message, "success");
                    var dialog = new Coral.Dialog();
                    dialog.id = 'dialogSuccess';
                    dialog.header.innerHTML = 'Success';
                    dialog.content.innerHTML = data.message;
                    dialog.footer.innerHTML = '<button class="ok-button" is="coral-button" variant="primary" icon="check" coral-close>OK</button>';
                    dialog.variant = 'success';
                    dialog.closable = "on";
                    dialog.show();
                    $(".modal").hide();
                }else{
                    ui.notify("Error", "Unable to process List Activation", "error");
                }
            }).fail(function (data) {
				$(".modal").hide();
                if (data && data.responseJSON && data.responseJSON.message){
                    ui.notify("Error", data.responseJSON.message, "error");
                }else{
                    ui.notify("Error", "Unable to process List Activation", "error");
                }
            });
        });
    });
})(window, document, $, Coral);

Once the code is built and deployed you will be able to access the tool by navigating the:

Tools -> {Tool Section} -> {Tool Name}

List Process Section

Now you can input the page paths as a line feed (no need to separate the paths using commas or any characters)

List Process Page

Upload the excel sheet or paths and select the queue method it’s recommended to use MCP Queue if you are activating more than 10K pages then it’s recommended to select MCP After 10K.

Select the Action to be performed.

List Replication Result
Page activation result

Working Code:
You can also access the working code on my GitHub repository link: https://github.com/kiransg89/aemoperations

Note: You can access this package over domain on AEM / AEMaaCS and over any environments and architects can handle the permissions by adding appropriate rep:policy on cq generator node or conf page

One thought on “AEM Tool – Publish / UnPublish / Delete List of pages

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s