Broken Page References AEM


Problem Statement:

How to get the list of all the broken references in AEM?


Requirement:

Get a List of all the broken references using MCP and provide the report


Introduction:

OOTB we get a Broken reference report provided by MCP, which can be used to get all the broken references in the content repo.

Broken Refernce Report

It’s highly recommended to run this process during

  1. off hours
  2. Don’t run on the root level
  3. Run it on 2nd level or 3rd level pages

How to run this process?

Provide Source path

Provide the regex so that it will consider only the references which point to /content or /etc (points to AEM)

You can also provide exclude properties to improve the traversal of nodes.

If you want to verify any broken links in the RTE fields or properties, then check the deep check checkbox and provide the properties list.

But the above process has a few issues.

  1. Html properties are not working as expected

We need a few customizations to this process by making a few changes to check HTML level references by adding JSOUP API

Add the following dependencies to your POM.xml

<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>com.adobe.acs</groupId>
    <artifactId>acs-aem-commons-bundle</artifactId>
    <scope>provided</scope>
</dependency>

Get the following Broken reference code into your local as shown below:

Add the following code as shown below:

if (htmlFields.contains(property.getKey())) {
            stream = stream.flatMap(val -> {
                try {
                    Document doc = Jsoup.parse(val);
                    Elements anchors = doc.select("a");
                    return anchors.stream().map(link -> link.attr("href"));
                } catch (Exception e) {
                    log.warn("Could not parse links from property value of {}", property.getKey(), e);
                    return Stream.empty();
                }
            });
        }
At Line number 207

When we run it on wknd site it would look something like this:

Broken Reference Report

Broken Asset References AEM


Problem Statement:

Get a List of all the Assets which are missing references


Requirement:

Get the list of broken asset references to unpublish and remove them repo to improve the system stability and performance.


Introduction:

How do assets get published?

  1.  The author uploads the images and publishes the assets
  2. Create a launcher and workflow which process assets metadata and publish the pages
  3. Whenever we publish any pages and if the page has references to assets, then during publishing, it asks to replicate the references as well.

What happens when the page is unpublished?

  1.  When the page is deactivated, assets referenced to the page will not be deactivated because this asset might have reference to the other pages hence out of the box assets won’t be deactivated.
  2. If we perform cleanup, deactivate and delete old pages, we might not be cleaning up assets related to this page.

Advantages of cleaning up old assets?

  1. Drastically reduces repository size
  2. Improves DAM Asset search
  3. Improves indexing

Get Publish Report using Assets Report:

  • Go to Tools -> Assets -> Reports as shown below:
Asset Reports
  • Click on create and click on Publish report
Select Publish report
  • Provide folder path and start date and end date
Add Report details
  • Select the columns as per requirement
Configure columns for the report
  • Finally, report will be ready with all the assets lists as shown below
Completed Reports
  • Download the report to see the final list of images
Example Report CSV file

If Images are unpublished then we can ask authors to review and delete them

If images are published but has no references to figure this out, we need a new process.

Broken Asset reference:

Add the following dependencies to your pom.xml

        <dependency>
            <groupId>org.jetbrains</groupId>
            <artifactId>annotations</artifactId>
            <version>17.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.adobe.acs</groupId>
            <artifactId>acs-aem-commons-bundle</artifactId>
            <version>5.2.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.jsoup</groupId>
            <artifactId>jsoup</artifactId>
            <version>1.13.1</version>
            <scope>provided</scope>
        </dependency>

Create a new MCP process by calling the MCP service and providing the implementation class:

package com.mysite.core.mcp;

import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import com.adobe.acs.commons.mcp.ProcessDefinitionFactory;
import com.day.cq.replication.Replicator;

@Component(service = ProcessDefinitionFactory.class)
public class BrokenAssetsFactory extends ProcessDefinitionFactory<BrokenAssets> {

    @Reference
    Replicator replicator;

    @Override
    public String getName() {
        return "Broken Asset References";
    }

    @Override
    public BrokenAssets createProcessDefinitionInstance() {
        return new BrokenAssets(replicator);
    }
}

Create an implementation class as shown below:

package com.mysite.core.mcp;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.EnumMap;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;
import javax.jcr.RepositoryException;
import org.apache.commons.collections4.ListUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.util.Text;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.LoginException;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.jetbrains.annotations.NotNull;
import com.adobe.acs.commons.data.CompositeVariant;
import com.adobe.acs.commons.data.Spreadsheet;
import com.adobe.acs.commons.fam.ActionManager;
import com.adobe.acs.commons.mcp.ProcessDefinition;
import com.adobe.acs.commons.mcp.ProcessInstance;
import com.adobe.acs.commons.mcp.form.Description;
import com.adobe.acs.commons.mcp.form.FileUploadComponent;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.RadioComponent;
import com.adobe.acs.commons.mcp.model.GenericReport;
import com.adobe.acs.commons.mcp.model.ManagedProcess;
import com.day.cq.replication.ReplicationStatus;
import com.day.cq.replication.Replicator;

/**
 * Relocate Pages and/or Sites using a parallelized move process
 */
public class BrokenAssets extends ProcessDefinition {
	
	private static final String SOURCE_PATH = "Path";
	private static final String CONTAINS_QUERY = "CONTAINS(s.*, '%s') or ";
	private final GenericReport report = new GenericReport();
	List<EnumMap<ReportColumns, String>> reportData = Collections.synchronizedList(new ArrayList<>());
	
	List<String> assetsList = new LinkedList<>();
	
    public enum PublishMethod {
        @Description("Select this option to generate Broken References Report")
        BROKEN_REFERENCE_REPORT
    }
    
    Replicator replicatorService;
    
    @FormField(name = "Mapping Excel", component = FileUploadComponent.class)
    private RequestParameter mappingExcel;
    private Spreadsheet spreadsheet;
    
    @FormField(name = "Type",
            description = "Please select the oprporiate option to execute",
            component = RadioComponent.EnumerationSelector.class,
            required = true,
            options = {"vertical", "default=BROKEN_REFERENCE_REPORT"})
    public PublishMethod publishMethod = PublishMethod.BROKEN_REFERENCE_REPORT;

	@FormField(name="Content Path",
			description="Content Path for search results",
			hint="/content",
			options={"default=/content"})
	public String contentPath = "/content";
    
    @FormField(name="Chunk count",
            description="Max number of chunk the search results",
            hint="4500",
            options={"default=3000"})
    public int chunkCount = 3000;
    
    @FormField(name="Retries",
            description="Max number of retries per commit",
            hint="2",
            options={"default=2"})
    public int retryCount = 2;
    
    @FormField(name="Retry delay",
            description="Delay between retries (in milliseconds)",
            hint="500,1000,...",
            options={"default=500"})
    public int retryWait = 500;
    
    public BrokenAssets(Replicator replicator) {
    	replicatorService = replicator;
	}

    @Override
    public void init() throws RepositoryException {    	
    	if(null != mappingExcel && mappingExcel.getSize() > 0) {
    		try {
                // Read spreadsheet
        		spreadsheet = new Spreadsheet(mappingExcel, SOURCE_PATH).buildSpreadsheet();
        		spreadsheet.getDataRowsAsCompositeVariants().forEach(row -> assetsList.add(getString(row, SOURCE_PATH)) );
            } catch (IOException e) {
                throw new RepositoryException("Unable to process spreadsheet", e);
            }
    	}    	    	    	
    }

    ManagedProcess instanceInfo;

    @Override
    public void buildProcess(ProcessInstance instance, ResourceResolver rr) throws LoginException, RepositoryException {
        instanceInfo = instance.getInfo();
        switch (publishMethod.name().toLowerCase()) {
	        case "broken_reference_report":
	        	instance.getInfo().setDescription("Collecting references");
	        	instance.defineAction("Searching Refs", rr, this::collectRefernce);
	            break;
        }        
    }

    @Override
    public void storeReport(ProcessInstance instance, ResourceResolver rr) throws RepositoryException, PersistenceException {    	
        report.setRows(reportData, ReportColumns.class);
        report.persist(rr, instance.getPath() + "/jcr:content/report");
    }

    public enum ReportColumns {
    	ASSET_PATH
    }
            
    protected void collectRefernce(ActionManager manager) {
    	manager.deferredWithResolver(this::collectReferences); 
    }
    
    private void collectReferences(ResourceResolver resourceResolver) throws Exception {    	
		if(!assetsList.isEmpty()) {						 
			RetryUtils.withRetry(retryCount, retryWait, () -> {
			    List<List<String>> chunkedAssetList = ListUtils.partition(assetsList, chunkCount);					
			    int counter = 0;
			    while(!chunkedAssetList.isEmpty() && counter < chunkedAssetList.size()) {
			        List<String> chunk = chunkedAssetList.get(counter);
			        counter++;
			        @NotNull Iterator<Resource> resourceResults = buildSQLQueryAndFetchResults(resourceResolver, contentPath, chunk);
			        collectReferences(resourceResults, chunk);
			    }
			});	
			if(!assetsList.isEmpty()) {
				assetsList.stream().forEach(this::reportResult);
			}
		}	        			
    }

	private void collectReferences(@NotNull Iterator<Resource> resourceResults, List<String> chunk) {
		while(resourceResults.hasNext()) {
			Resource resultRes = resourceResults.next();
			if (null != resultRes) {
				resultRes.getValueMap().entrySet().forEach(property -> {
					if(!property.getKey().equalsIgnoreCase("dam:folderThumbnailPaths")) {
						Object prop = property.getValue();
						if (prop.getClass() == String[].class) {
							List<String> propertyValue = Arrays.asList((String[]) prop);
							if (!propertyValue.isEmpty()) {
								List<String> matchingAsset = chunk.stream().filter(
										assetPat -> propertyValue.stream().anyMatch(sam -> sam.contains(assetPat)))
										.collect(Collectors.toList());
								if(!matchingAsset.isEmpty()) {
									chunk.removeIf(matchingAsset::contains);
									assetsList.removeIf(matchingAsset::contains);
								}
							}
						} else if (prop.getClass() == String.class) {
							String propertyValue = (String) prop;
							if (StringUtils.isNotEmpty(propertyValue) ) {
								Optional<String> matchingAsset = chunk.stream().filter(propertyValue::contains).findAny();
								if(matchingAsset.isPresent()) {
									chunk.remove(matchingAsset.get());
									assetsList.remove(matchingAsset.get());
								}
							}
						}
					}
				});
			}
		}
	}

	private void reportResult(String path) {
		EnumMap<ReportColumns, String> row = new EnumMap<>(ReportColumns.class);
		row.put(ReportColumns.ASSET_PATH, path);		
		reportData.add(row);
	}	
	
	private @NotNull Iterator<Resource> buildSQLQueryAndFetchResults(ResourceResolver resolver, String contentPath, List<String> chunk) {
        String groupStr = chunk.stream().map(r -> String.format(CONTAINS_QUERY, Text.escapeIllegalXpathSearchChars(r).replaceAll("'", "''"))).collect(Collectors.joining());        
		String querySt = "SELECT * FROM [nt:base] AS s WHERE ISDESCENDANTNODE(["+contentPath+"]) and "+ StringUtils.substringBeforeLast(groupStr, "or");
		return resolver.findResources(querySt, "JCR-SQL2");
    }
	
	@SuppressWarnings({"rawtypes", "unchecked"})
    private String getString(Map<String, CompositeVariant> row, String path) {
        CompositeVariant v = row.get(path.toLowerCase(Locale.ENGLISH));
        if (v != null) {
            return (String) v.getValueAs(String.class);
        } else {
            return null;
        }
    }
}
package com.mysite.core.mcp;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryUtils {
    private static final Logger log = LoggerFactory.getLogger(RetryUtils.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int i = 0; i < maxTimes; i++) {
            try {
                call.process();
                return true;
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (i + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

Running Asset Reference Process:

After building the code you can see the new Process showing up in MCP

Borken Asset Refernce Process

Copy the Path column into the new Excel sheet as shown below

Path column into new excel file

Upload into the process and start to see all the images which are published yet unreferenced as shown below

Why does Chunk count?

Chunk count helps the SQL 2 query to group by the paths, which will be maxing 4500 and it won’t take more than that (configurable based on the environment). However basically, if we have 20000 / 4500 = 4.44 ~ 5 we will be running the query max five times to generate the below report

Share the report with the content authors team to validate if images are required if not plan to clean up using

After completing the process
Example report and we can download and share with Authors

Clean up Process:

Authors don’t have to unpublish and delete individual images, then can you use the below process to upload the excel sheet with all the approved image paths and upload it to the process to deactivate and delete

AEM Publish / UnPublish / Delete List of pages – MCP Process

Unclosed resource resolver complete guide AEM


Problem Statement:

Unclosed resource resolver issue is causing performance impact on the AEM environment

It’s hard to debug and fix the resolver issues

Best practices to close the resolver


Requirement:

Provide all the best practices to close the unclosed resource resolve and in turn improve environment stability.


Introduction:

Usually, we create/open a resource resolver:

  1. Servlets
  2. OSGi Services
  3. Workflows
  4. Schedulers

Purpose of service user based resolver.

To get access to the paths which is blocked for every one group, make changes and commit changes etc.

Creating Resource resolver instance:

This feature was released with Java 7 and the try-with-resources statement is a try statement that declares one or more resources. A resource is an object that must be closed after the program is finished with it. The try-with-resources statement ensures that each resource is closed at the end of the statement. Any object that implements java.lang.AutoCloseable, which includes all objects which implement java.io.Closeable, can be used as a resource.

So basically, creating the resolver within the calling class like below will auto close:

try (ResourceResolver resourceResolver = resolverFactory.getServiceResourceResolver(Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "{service user name}"))){
    //business logic
}

And it is highly recommended to use the following APIs as well:

  1. FileReader
  2. ZipFile
  3. BufferedWriter

etc.

Closing the resolver for Queries:

Whenever we come across queries, we need to close the resolver manually even after using the try resource approach:

Try resource Approach:

ResourceResolver resolverResolverLeakingReference = null;
try (ResourceResolver resolver = resourceResolverFactory.getServiceResourceResolver(Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "{service user name}"))) {
    session = resolver.adaptTo(Session.class);
    queryBuilder = resolver.adaptTo(QueryBuilder.class);
    
    PredicateGroup predicateGroup = null;
    try {
        predicateGroup = PredicateGroup.create(map);
    } catch (VerifyError ex) {
        predicateGroup = new PredicateGroup();
    }

    Query query = queryBuilder.createQuery(predicateGroup, session);
    SearchResult result = query.getResult();
    for (final Hit hit : result.getHits()) {
        Resource resource = hit.getResource();            	
        if(resolverResolverLeakingReference == null){
            resolverResolverLeakingReference =  resource.getResourceResolver();
        }    
        //buisness logic                            
    }
} catch (Exception e) {
    log.error("Error fetching locations count results", e);
} finally {
    if (resolverResolverLeakingReference != null) {
        // Always Close the leaking QueryBuilder resourceResolver.
        resolverResolverLeakingReference.close();    
    }        
}

Requet.getResolver Approach:

ResourceResolver leakingResourceResolverReference = null;
try {
    Session session = request.getResourceResolver().adaptTo(Session.class);
    Map<String, String> map = new HashMap<>();
    //map put goes here
        Query query = queryBuilder.createQuery(PredicateGroup.create(map), session);
        SearchResult result = query.getResult();
        for (Hit hit : result.getHits()) {
            if(leakingResourceResolverReference == null) {
                leakingResourceResolverReference =  hit.getResource().getResourceResolver();
            }		         
            //buisness logic
        }    
} catch(Exception e) {
    log.info("error::{}",e.getMessage());
} finally {
    if(leakingResourceResolverReference != null){
        leakingResourceResolverReference.close();
    }
}

Or you can also follow this:

SearchResult result;
try {
    Session session = request.getResourceResolver().adaptTo(Session.class);
    Map<String, String> map = new HashMap<>();
    //map put goes here
        Query query = queryBuilder.createQuery(PredicateGroup.create(map), session);
        result = query.getResult();
        for (Hit hit : result.getHits()) {    
            //buisness logic
        }    
} catch(Exception e) {
    log.info("error::{}",e.getMessage());
} finally {
    Iterator<Resource> resources = result.getResources();
    if (resources.hasNext()) {
        resources.next().getResourceResolver().close();
    }
}

Avoid query builder and use SQL2:

try (ResourceResolver resourceResolver = resolverFactory.getServiceResourceResolver(Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "{service user name}"))){
    String querySt = "SELECT * FROM [nt:base] ";		
    Iterator<Resource> results = resourceResolver.findResources(querySt, "JCR-SQL2");
    //buisness logc
}

Use Java Streams:

You can follow the below blog to use Java stream for executing queries in AEM

AEM Query builder using Java streams

Purge Old CRX Packages AEM


Problem statement:

AEM environment size is increasing because of user-generated packages


Requirement:

Can we purge all the user-generated packages to improve stability?


Introduction:

A package is a zip file holding repository content in the form of a file-system serialization (called “vault” serialization). This provides an easy-to-use-and-edit representation of files and folders.

Packages include content, both page content and project-related content, selected using filters.

A package also contains vault meta information, including the filter definitions and import configuration information. Additional content properties (that are not used for package extraction) can be included in the package, such as a description, a visual image, or an icon; these properties are for the content package consumer and for informational purposes only.

Usually, Developers or AEM content synch or even code deployment will keep on piling up in the CRX packages and it will be consuming spaces on MBs and even sometimes GBs.

If we move more packages, then loading crx/packmgr would more time.

Hence you can create a scheduler that runs on off-hours which cleans up the packages and which will get back the space and avoids extra maintenance tasks.

The below scheduler will remove all the packages for my_package group we can add business logic to handle for other groups

package com.mysite.core.schedulers;

import org.osgi.service.metatype.annotations.AttributeDefinition;
import org.osgi.service.metatype.annotations.AttributeType;
import org.osgi.service.metatype.annotations.ObjectClassDefinition;

@ObjectClassDefinition(name = "Old Packages Purge Schedular", description = "Remove old packages from different paths")
public @interface PurgeOldPackagesSchedulerConfig {

    String DEFAULT_SCHEDULER_EXPRESSION = "0 0 16 ? * SUN *"; // every Sunday 4 PM
    boolean DEFAULT_SCHEDULER_CONCURRENT = false;

    @AttributeDefinition(name = "Enabled", description = "True, if scheduler service is enabled", type = AttributeType.BOOLEAN)
    boolean enabled() default true;

    @AttributeDefinition(name = "Cron expression defining when this Scheduled Service will run", description = "[every minute = 0 * * * * ?], [12:00am daily = 0 0 0 ? * *]", type = AttributeType.STRING)
    String schedulerExpression() default DEFAULT_SCHEDULER_EXPRESSION;

    @AttributeDefinition(name = "package paths", description = "package folder paths", type = AttributeType.STRING)
    String[] packagesPaths() default {"my_packages"};
}
package com.mysite.core.schedulers;

import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import org.apache.jackrabbit.vault.packaging.JcrPackage;
import org.apache.jackrabbit.vault.packaging.JcrPackageManager;
import org.apache.jackrabbit.vault.packaging.Packaging;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceResolverFactory;
import org.apache.sling.commons.scheduler.ScheduleOptions;
import org.apache.sling.commons.scheduler.Scheduler;
import org.osgi.service.component.annotations.Activate;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.ConfigurationPolicy;
import org.osgi.service.component.annotations.Deactivate;
import org.osgi.service.component.annotations.Modified;
import org.osgi.service.component.annotations.Reference;
import org.osgi.service.metatype.annotations.Designate;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Component(immediate = true, service = PurgeOldPackagesScheduler.class, configurationPolicy = ConfigurationPolicy.REQUIRE)
@Designate(ocd = PurgeOldPackagesSchedulerConfig.class)
public class PurgeOldPackagesScheduler implements Runnable {

    private static final Logger log = LoggerFactory.getLogger(PurgeOldPackagesScheduler.class);
    /**
     * Id of the scheduler based on its name
     */
    private String schedulerJobName;

    Session session;
    private List<String> packagesPathsList;

    SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");

    @Reference
    private Packaging packaging;

    @Reference
    private Scheduler scheduler;

    @Reference
    private ResourceResolverFactory resolverFactory;

    @Activate
    @Modified
    protected void activate(PurgeOldPackagesSchedulerConfig purgeOldPackagesSchedulerConfig) {
        /**
         * Creating the scheduler id
         */

        this.schedulerJobName = this.getClass().getSimpleName();
        addScheduler(purgeOldPackagesSchedulerConfig);
        packagesPathsList = Arrays.asList(purgeOldPackagesSchedulerConfig.packagesPaths());
    }

    /**
     * @see Runnable#run().
     */
    @Override
    public final void run() {
        log.debug("PurgeOldPackagesScheduler Job started");
        purgeOldPackages(packagesPathsList);
        log.debug("PurgeOldPackagesScheduler Job completed");
    }

    void purgeOldPackages(List<String> packagesPathsList) {
        try (ResourceResolver resourceResolver = resolverFactory.getServiceResourceResolver(Collections
                .singletonMap(ResourceResolverFactory.SUBSERVICE, ServiceUserConstants.ADMINISTRATIVE_SERVICE_USER))) {
            session = resourceResolver.adaptTo(Session.class);
            JcrPackageManager jcrPackageManager = packaging.getPackageManager(session);
            packagesPathsList.forEach(group -> {
                removePackages(jcrPackageManager, 0, group);
            });
        } catch (Exception e) {
            log.error(e.getMessage(), e);
        } finally {
            if (session != null) {
                session.logout();
            }
        }
    }

    @Deactivate
    protected void deactivate() {

        /**
         * Removing the scheduler
         */
        removeScheduler();
    }

    /**
     * This method adds the scheduler
     *
     * @param purgeOldPackagesSchedulerConfig
     */
    private void addScheduler(PurgeOldPackagesSchedulerConfig purgeOldPackagesSchedulerConfig) {

        /**
         * Check if the scheduler is enabled
         */
        if (purgeOldPackagesSchedulerConfig.enabled()) {

            /**
             * Scheduler option takes the cron expression as a parameter and run accordingly
             */
            ScheduleOptions scheduleOptions = scheduler.EXPR(purgeOldPackagesSchedulerConfig.schedulerExpression());

            /**
             * Adding some parameters
             */
            scheduleOptions.name(schedulerJobName);
            scheduleOptions.canRunConcurrently(false);

            /**
             * Scheduling the job
             */
            scheduler.schedule(this, scheduleOptions);

            log.info("{} Scheduler added", schedulerJobName);
        } else {
            log.info("Scheduler is disabled");
            removeScheduler();
        }

    }

    /**
     * This method removes the scheduler
     */
    private void removeScheduler() {

        log.info("Removing scheduler: {}", schedulerJobName);

        /**
         * Unscheduling/removing the scheduler
         */
        scheduler.unschedule(String.valueOf(schedulerJobName));
    }

    private void removePackages(JcrPackageManager jcrPackageManager, int counterStart, String groupName) {
        try {
            List<JcrPackage> packages = jcrPackageManager.listPackages(groupName, false);
            AtomicInteger counter = new AtomicInteger(counterStart);
            if (null != packages && !packages.isEmpty()) {
                packages.stream().sorted(Comparator.nullsLast((e1, e2) -> {
                    try {
                        return ((JcrPackage) e1).getPackage().getCreated()
                                .compareTo(((JcrPackage) e2).getPackage().getCreated());
                    } catch (RepositoryException | IOException ex) {
                        log.error(ex.getMessage());
                    }
                    return -1;
                }).reversed()).forEach(pack -> {
                    if (counter.incrementAndGet() > 3) {
                        try {
                            jcrPackageManager.remove(pack);
                        } catch (RepositoryException ex) {
                            log.error(ex.getMessage());
                        }
                    }
                });
            }
        } catch (RepositoryException e) {
            e.printStackTrace();
        }
    }
}

Caching AEM GraphQL queries with content fragment


Problem Statement:

How can I persist query?

How to cache my query results?
How to Update my queries?


Requirement:

Provide details on how to add the persist graphql query, cache the results from graphql and update the persisted query

Provide curl commands to execute in terminal or on postman


Introduction:

Persisted Queries (Caching)

After preparing a query with a POST request, it can be executed with a GET request that can be cached by HTTP caches or a CDN.

This is required as POST queries are usually not cached, and if using GET with the query as a parameter there is a significant risk of the parameter becoming too large for HTTP services and intermediates.

Persisted queries must always use the endpoint related to the appropriate Sites configuration; so, they can use either, or both:

  • Specific Sites configuration and endpoint

Creating a persisted query for a specific Sites configuration requires a corresponding Sites-configuration-specific endpoint (to provide access to the related Content Fragment Models).

For example, to create a persisted query specifically for the SampleGraphQL Sites configuration:

a corresponding SampleGraphQL-specific Sites configuration

  • Go to the tools section for the aem and general section and select Configuration Browser as shown below
Configuration browser
  • Add select the conf folder and go to the properties and make GraphQL Persistent Queries checkbox is checked
Enable persistent queries

a SampleGraphQL-specific endpoint must be created in advance.

  • Go to tools section for the aem and assets section and select GraphQL as shown below
assets -> graphql
  • Add the new end point as shown below:
endpoint

Add the following CORS configurations for the GraphQL API calls:

CORS config

Register graphql search path:

Register Servlet path

Here are the steps required to persist a given query:

Prepare the query by putting it to the new endpoint URL /graphql/persist.json/<config>/<persisted-label>.

For example, create a persisted query:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/cities' \
--header 'Content-Type: application/json' \
--data-raw '{
  cityList {
    items {
      _path
      name
      country
      population
    }
  }
}'
  • At this point, check the response.

For example, check for success:

{
    "action": "create",
    "configurationName": "SampleGraphQL",
    "name": "cities",
    "shortPath": "/SampleGraphQL/cities",
    "path": "/conf/SampleGraphQL/settings/graphql/persistentQueries/cities"
}

You can then replay the persisted query by getting the URL /graphql/execute.json/<shortPath>.

For example, use the persisted query:

curl -u admin:admin -X GET 'http://localhost:4502/graphql/execute.json/SampleGraphQL/cities' \
--header 'Authorization: Basic YWRtaW46YWRtaW4='

Update a persisted query by POSTing to an already existing query path.

For example, use the persisted query:

curl -u admin:admin -X POST 'http://localhost:4502/graphql/persist.json/SampleGraphQL/cities' \
--header 'Content-Type: application/json' \
--data-raw '{
  cityList {
    items {
      _path
      name
      country
      population
    }
  }
}'

Create a wrapped plain query.

For example:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/plain-cities' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "{cityList { items { _path name country country population } } }"}'

Create a wrapped plain query with cache control.

For example:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/plain-cities-max-age' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "{cityList { items { _path name country country population } } }", "cache-control": { "max-age": 300 }}'

Create a persisted query with parameters:

For example:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/plain-cities-query-parameters' \
--header 'Content-Type: application/json' \
--data-raw \
'query GetAsGraphqlModelTestByPath($apath: String!) {
    cityByPath(_path: $apath) {
        item {
        _path
        name
        country
        population
        }
    }
  }'

Executing a query with parameters.

For example:

curl -u admin:admin -X POST \
    -H "Content-Type: application/json" \
    "http://localhost:4502/graphql/execute.json/SampleGraphQL/plain-cities-query-parameters;apath=%2Fcontent%2Fdam%2Fsample-content-fragments%2Fcities%2Fberlin"

curl -u admin:admin -X GET \
    "http://localhost:4502/graphql/execute.json/SampleGraphQL/plain-cities-query-parameters;apath=%2Fcontent%2Fdam%2Fsample-content-fragments%2Fcities%2Fberlin"

AEM Content Fragments with GraphQL – Getting started with GraphQL

Problem Statement:

What is GraphQL?

How GraphQL can be used with Content Fragments?

Introduction:

What is GraphQL?

GraphQL is a query language for APIs and provides a complete and understandable description of the data in your API.

For example:

Let’s consider an external system with the following tables:

1 to 1 relationship between Company table with person table and 1 to 1 person table with awards table.

If I need to get all awards

You might be doing a call to

{domain}/api/awards

awards API flow

To get individual person and awards

{domain}/api/persons?personID{ID}&awards={ID}

person API flow

To get individual company and person and awards

{domain}/api/company?companyNam={NAME}personID={ID}&awards={ID}

Company API flow

But in GraphQL you can send the parameters like a query and get all the related content as well

GraphQL API flow

To use Graph QL you need to prepare schemas and based on the schema you can do filter the data.

For more information on GraphQL, you can be visiting the link

Benefits:

  • Avoiding iterative API requests as with REST,
  • Ensuring that delivery is limited to the specific requirements,
  • Allowing for bulk delivery of exactly what is needed for rendering as the response to a single API query.

How GraphQL can be used with Content Fragments?

GraphQL is a strongly typed API, which means that data must be clearly structured and organized by type.

The GraphQL specification provides a series of guidelines on how to create a robust API for interrogating data on a certain instance. To do this, a client needs to fetch the Schema, which contains all the types necessary for a query.

For Content Fragments, the GraphQL schemas (structure and types) are based on Enabled Content Fragment Models and their data types.

Content Fragments can be used as a basis for GraphQL for AEM queries as:

  • They enable you to design, create, curate and publish page-independent content.
  • The Content Fragment Models provide the required structure by means of defined data types.
  • The Fragment Reference, available when defining a model, can be used to define additional layers of structure.
Model References provided by Adobe

Content Fragments

  • Contain structured content.
  • They are based on a Content Fragment Model, which predefines the structure for the resulting fragment.

Content Fragment Models

  • Are used to generate the Schemas, once Enabled.
  • Provide the data types and fields required for GraphQL. They ensure that your application only requests what is possible, and receives what is expected.
  • The data type Fragment References can be used in your model to reference another Content Fragment, and so introduce additional levels of structure.

Fragment References

  • Is of particular interest in conjunction with GraphQL.
  • Is a specific data type that can be used when defining a Content Fragment Model.
  • References another fragment, dependent on a specific Content Fragment Model.
  • Allows you to retrieve structured data.
    • When defined as a multifeed, multiple sub-fragments can be referenced (retrieved) by the prime fragment.

JSON Preview

To help with designing and developing your Content Fragment Models, you can preview JSON output.

Install:

  1. AEM 6.5.11 (aem-service-pkg-6.5.11.zip)
  2. Graph QL OAK Index (cfm-graphql-index-def-1.0.0.zip)
  3. GraphiQL Developer tool (graphiql-0.0.6.zip)

For AEMacS you will get the content fragment with the latest update.

Go to configuration folder

  1. AEM tools section
  2. General selection in sidebar
  3. Configuration bowser

As shown below:

Configuration Folder

Create a configuration folder and select

  1. Content Fragment Models
  2. GraphQL Persistent Queries

As shown below:

Create a Conf folder with required checkboxes

Go to Assets Model:

  1. AEM tools section
  2. Assets selection in sidebar
  3. Content Fragments Model

As shown below:

Go to Assets CF

Select the folder and create the content fragments as shown below:

CF models

You can also install the package attached here

Go to the following URL to access the GraphiQL developer tool and run the following query:

Note: you can also get all the autosuggestions by using the ctrl+space shortcut

{
  cityByPath(_path: "/content/dam/sample-content-fragments/cities/berlin") {
    item {
      _path
      name
      country
      population
      categories
    }
  }
}
GraphiQL developer tool

Download Sample Package here

You can also find more queries and filters in the following link

AEM with Java streams

Problem statement:

How to use java streams in AEM? Can I use streams for iterating and resources?

Requirement:

Use Java streams to iterate child nodes, validating and resources and API’s.

Introduction:

There are a lot of benefits to using streams in Java, such as the ability to write functions at a more abstract level which can reduce code bugs, compact functions into fewer and more readable lines of code, and the ease they offer for parallelization

  • Streams have a strong affinity with functions
  • Streams encourage less mutability
  • Streams encourage looser coupling
  • Streams can succinctly express quite sophisticated behavior
  • Streams provide scope for future efficiency gains

Java Objects:

This class consists of static utility methods for operating on objects. These utilities include null-safe or null-tolerant methods for computing the hash code of an object, returning a string for an object, and comparing two objects.

if (Objects.nonNull(resource)) {
  resource.getValueMap().get("myproperty", StringUtils.EMPTY);
}

Java Optional:

Trying using Java Optional util, which is a box type that holds a reference to another object.

Is immutable and non serializable ant there is no public constructor and can only be present or absent

It is created by the of(), ofNullable(), empty() static method.

In the below example Optional resource is created and you can check whether the resource is present and if present then get the valuemap

Optional < Resource > res = Optional.ofNullable(resource);
if (res.isPresent()) {
    res.get().getValueMap().get("myproperty", StringUtils.EMPTY);
}

you can also call stream to get children’s as shown below:

Optional < Resource > res = Optional.ofNullable(resource);
if (res.isPresent()) {
    List < Resource > jam = res.stream().filter(Objects::nonNull).collect(Collectors.toList());
}

Java Stream Support:

Low-level utility methods for creating and manipulating streams. This class is mostly for library writers presenting stream views of data structures; most static stream methods intended for end users are in the various Stream classes.

In the below example we are trying to get a resource iterator to get all the child resources and map the resources to a page and filter using Objects and finally collect the list of pages.

Iterator < Resource > iterator = childResources.getChildren().iterator();
List < Page > pages = StreamSupport.stream(((Iterable < Resource > )() -> iterator).spliterator(), false)
  .map(currentPage.getPageManager()::getContainingPage).filter(Objects::nonNull)
  .collect(Collectors.toList());

We can also Optional utility to get the children resources or empty list to avoid all kinds of null pointer exceptions.

List < Resource > pagesList = Optional.ofNullable(resource.getChild(Teaser.NN_ACTIONS))
  .map(Resource::getChildren)
  .map(Iterable::spliterator)
  .map(s -> StreamSupport.stream(s, false))
  .orElseGet(Stream::empty)
  .collect(Collectors.toList());

We can also adapt the resource to Page API and call the listchilderens to get all the children and using stream support we are going to map the page paths into a list as shown below:

terator < Page > childIterator = childResources.adaptTo(Page.class).listChildren();
StreamSupport.stream(((Iterable < Page > )() -> childIterator).spliterator(), false)
  .filter(Objects::nonNull)
  .map(childPage -> childPage.getPath())
  .collect(Collectors.toList());

Does this works only on resource and page API?

No, we can also use Content Fragment and other API’s as well for example in the below code we are trying to iterate contentfragment and get all the variations of the contentfragment.

Optional < ContentFragment > contentFragment = Optional.ofNullable(resource.adaptTo(ContentFragment.class));
Iterator < VariationDef > versionIterator = contentFragment.get().listAllVariations();
List < String > variationsList = StreamSupport.stream(((Iterable < VariationDef > )() -> versionIterator).spliterator(), false)
  .filter(Objects::nonNull)
  .map(cfVariation -> cfVariation.getTitle())
  .collect(Collectors.toList());

You can also learn more about other tricks and techniques of Java Streams:

AEM Query builder using Java streams

AEM Get linked Content fragments content

Problem Statement:

We have the following content fragment as part of the AEM

  1. Car details
  2. Agent details

And each car can have multiple agents or agents will be selling multiple cars. How can we link between Cars and Agents CF? and how can we get the linked content onto the page?

Requirement:

Link the Car and Agent CF to maintain the relationship between the content fragments and the same can be pulled into the page and exported.

Introduction:

Content Fragments (CF) allow us to manage page-independent content. They help us prepare content for use in multiple locations/over multiple channels. These are text-based editorial content that may include some structured data elements that are considered pure content without design or layout information. Content Fragments are intended to be used and reused across channels.

Usage

  • Highly structured data-entry/form-based content
  • Long-form editorial content (multi-line elements)
  • Content managed outside the life cycle of the channels delivering it

Create the Car and Agents content fragment models as shown below:

Agent Content fragment
Car Content Fragment

Create a custom component called has linkedcontentfragment as shown below:

Based on the element name condition call the LinkedContentFragment Sling model and also pass the elements to be rendered (based on element names, element data will be pulled).

<template data-sly-template.element="${@ element='The content fragment element'}">
    <div class="cmp-contentfragment__element cmp-contentfragment__element--${element.name}" data-cmp-contentfragment-element-type="${element.dataType}">
        <dd class="cmp-contentfragment__element-value">
            <sly data-sly-test="${element.dataType == 'calendar'}" data-sly-use.tpl="core/wcm/components/contentfragment/v1/contentfragment/calendar.html"
                 data-sly-call="${tpl.element @ date = element.value}"></sly>
            <sly data-sly-test="${element.dataType == 'boolean'}">${element.value ? "true" : "false"}</sly>
            <sly data-sly-test="${element.dataType != 'calendar' && element.dataType != 'boolean' && element.name != 'agentDetails'}">${(element.value) @join='<br/>', context='html'}</sly>
            <sly data-sly-test="${element.name == 'agentDetails'}"
                 data-sly-use.linkedFragment="${'com.mysite.core.models.LinkedContentFragment' @elementValue=element, elements='agentTitle,agentDescription,agentAddress'}">
                data-sly-list.agentData="${linkedFragment.agentsList}">
                <dl data-cmp-data-layer="${agentData.data.json}"
                    data-sly-list.element="${agentData.elements}"
                    data-sly-use.elementTemplate="mysite/components/linkedcontentfragment/element.html">
                    <sly data-sly-call="${elementTemplate.element @ element=element}">
                    </sly>
                </dl>
            </sly>
        </dd>
    </div>
</template>

Create Sling model interface LinkedContentFragment as shown below:

package com.mysite.core.models;

import com.adobe.cq.wcm.core.components.models.Component;
import com.adobe.cq.wcm.core.components.models.contentfragment.ContentFragment;
import java.util.List;

/**
 * Defines the {@code Agent CF Model} Sling Model for the {@code /apps/mysite/components/linkedcontentfragment} component.
 */
public interface LinkedContentFragment extends Component {
    /**
     * Returns the Agents List
     *
     * @return the Content Fragment
     */
    default List<ContentFragment> getAgentsList() {
        throw new UnsupportedOperationException();
    }
}

Create model implementation class called LinkedContentFragmentImpl as shown below, get the element data (String array of paths) and elements to be pulled create a synthetic resource and adapt to core component Content fragment model to pull the element details as well as datalayer (tracking purposes)

package com.mysite.core.models.impl;

import com.adobe.cq.wcm.core.components.models.contentfragment.ContentFragment;
import com.adobe.cq.wcm.core.components.models.contentfragment.DAMContentFragment.DAMContentElement;
import com.adobe.cq.wcm.core.components.util.AbstractComponentImpl;
import com.adobe.granite.ui.components.ds.ValueMapResource;
import com.mysite.core.models.LinkedContentFragment;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ValueMap;
import org.apache.sling.api.wrappers.ValueMapDecorator;
import org.apache.sling.models.annotations.Model;
import org.apache.sling.models.annotations.injectorspecific.InjectionStrategy;
import org.apache.sling.models.annotations.injectorspecific.OSGiService;
import org.apache.sling.models.annotations.injectorspecific.RequestAttribute;
import org.apache.sling.models.annotations.injectorspecific.SlingObject;
import org.apache.sling.models.factory.ModelFactory;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;

@Model(adaptables = SlingHttpServletRequest.class, adapters = {LinkedContentFragment.class}, resourceType = LinkedContentFragmentImpl.RESOURCE_TYPE)
public class LinkedContentFragmentImpl extends AbstractComponentImpl implements LinkedContentFragment {

    public static final String RESOURCE_TYPE = "mysite/components/linkedcontentfragment";
    private static final String CF_DISPLAY_MODE = "displayMode";
    private static final String CF_ELEMENTS = "elementNames";
    private static final String CF_FRAGMENT_PATH = "fragmentPath";

    @RequestAttribute(injectionStrategy = InjectionStrategy.OPTIONAL)
    private DAMContentElement elementValue;

    @RequestAttribute(injectionStrategy = InjectionStrategy.OPTIONAL)
    private String elements;

    @SlingObject
    private ResourceResolver resourceResolver;

    @OSGiService
    private ModelFactory modelFactory;

    @Override
    public List<ContentFragment> getAgentsList() {
        String[] elementList = elements.split(",");
        String[] agentPaths = elementValue.isMultiValue() ? (String[]) elementValue.getValue() : new String[] { (String) elementValue.getValue() };
        List<ContentFragment> elementsFragmentList = new ArrayList<>();
        Arrays.stream(agentPaths).forEach(agentPath -> {
            elementsFragmentList.add(modelFactory.getModelFromWrappedRequest(request, createLinkedSyntheticResource(agentPath, elementList), ContentFragment.class));
        });
        return elementsFragmentList;
    }

    private ValueMapResource createLinkedSyntheticResource(String path, String ... elementList) {
        ValueMap properties = new ValueMapDecorator(new HashMap<>());
        properties.put(CF_ELEMENTS, elementList);
        properties.put(CF_DISPLAY_MODE, "multi");
        properties.put(CF_FRAGMENT_PATH, path);
        return new ValueMapResource(resourceResolver, resource.getPath(), RESOURCE_TYPE, properties);
    }
}

Once we author the car model, we will be pulling the linked agent details as well as shown below:

Component Authoring
Content Fragment rendering

You can also refer to the below link to download the working code

https://github.com/kiransg89/LinkedContentFragment

AEM Content Fragment with Image support

Problem Statement:

How can I get support for Core Image component within content fragment component?

Why should we use Core Image component?

Requirement:

Get support for OOTB image component within content fragment component

Introduction:

The Image Component features adaptive image selection and responsive behaviour with lazy loading for the page visitor as well as easy image placement.

The Image Component comes with robust responsive features ready right out of the box. At the page template level, the design dialog can be used to define the default widths of the image asset. The Image Component will then automatically load the correct width to display depending on the browser window’s size. As the window is resized, the Image Component dynamically loads the correct image size on the fly. There is no need for component developers to worry about defining custom media queries since the Image Component is already optimized to load your content.

In addition, the Image Component supports lazy loading to defer loading of the actual image asset until it is visible in the browser, increasing the responsiveness of your pages.

Create Custom Image content fragment component as shown below and add the following conditions into the element.html file:

Image Contentfragment Component

In the above HTL we are trying to tell if the element name contains the “image” (eg: primaryimage) keyword then don’t print it instead of that call an image model and get synthetic image resource

Note: Make sure your content fragment element name (field name) contains image word

Create a Sling mode ImageContentFragment Interface as shown below:

package com.mysite.core.models;

import org.apache.sling.api.resource.Resource;
import org.osgi.annotation.versioning.ConsumerType;

@ConsumerType
public interface ImageContentFragment {
    /**
     * Getter for Image Synthetic Resource
     *
     * @return resource
     */
    default Resource getImageResource() {
        throw new UnsupportedOperationException();
    }
}

Create a Sling mode implementation ImageContentFragmentImpl class as shown below, this class is used to create a synthetic image resource for Adaptive Image Servlet to work:

package com.mysite.core.models.impl;

import java.util.HashMap;
import javax.annotation.PostConstruct;
import com.mysite.core.models.ImageContentFragment;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ValueMap;
import org.apache.sling.api.wrappers.ValueMapDecorator;
import org.apache.sling.models.annotations.Model;
import org.apache.sling.models.annotations.injectorspecific.InjectionStrategy;
import org.apache.sling.models.annotations.injectorspecific.RequestAttribute;
import org.apache.sling.models.annotations.injectorspecific.SlingObject;
import org.apache.sling.models.annotations.injectorspecific.ValueMapValue;
import com.adobe.cq.wcm.core.components.models.contentfragment.ContentFragment;
import com.adobe.cq.wcm.core.components.models.contentfragment.DAMContentFragment.DAMContentElement;
import com.adobe.granite.ui.components.ds.ValueMapResource;
import com.day.cq.commons.DownloadResource;
import lombok.Getter;

@Model(adaptables = SlingHttpServletRequest.class, adapters = {ImageContentFragment.class}, resourceType = ImageContentFragmentImpl.RESOURCE_TYPE)
public class ImageContentFragmentImpl implements ImageContentFragment{

    public static final String RESOURCE_TYPE = "mysite/components/image";
    private static final String IMAGE = "/image";

    @SlingObject
    private ResourceResolver resourceResolver;

    @SlingObject
    protected Resource resource;

    @ValueMapValue(name = ContentFragment.PN_PATH, injectionStrategy = InjectionStrategy.OPTIONAL)
    private String fragmentPath;

    @RequestAttribute(injectionStrategy = InjectionStrategy.OPTIONAL)
    private DAMContentElement imageElement;

    @ValueMapValue(injectionStrategy = InjectionStrategy.OPTIONAL)
    private String fileReference;

    @Getter
    private Resource imageResource;

    @PostConstruct
    private void initModel() {
        if (StringUtils.isNotEmpty(fragmentPath) && null != imageElement && null != imageElement.getValue()) {
            createSyntheticResource(imageElement.getValue().toString(), resource.getPath() + IMAGE);
        }
    }

    private void createSyntheticResource(String imagePath, String path) {
        ValueMap properties = new ValueMapDecorator(new HashMap<>());
        properties.put(DownloadResource.PN_REFERENCE, imagePath);
        imageResource = new ValueMapResource(resourceResolver, path, ImageContentFragmentImpl.RESOURCE_TYPE, properties);
    }
}

Create an ImageDelegate Lombok based delegation class as shown below, for this example, I am using Image V3 component and this class is used to modify the image srcset method to add image path into the image URL:

package com.mysite.core.models.impl;

import com.adobe.cq.export.json.ExporterConstants;
import com.adobe.cq.wcm.core.components.models.Image;
import lombok.experimental.Delegate;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Exporter;
import org.apache.sling.models.annotations.Model;
import org.apache.sling.models.annotations.Via;
import org.apache.sling.models.annotations.injectorspecific.Self;
import org.apache.sling.models.annotations.injectorspecific.SlingObject;
import org.apache.sling.models.annotations.via.ResourceSuperType;
import org.apache.sling.servlets.post.SlingPostConstants;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;

@Model(adaptables = SlingHttpServletRequest.class, adapters = {Image.class}, resourceType = ImageDelegate.RESOURCE_TYPE, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
@Exporter(name = ExporterConstants.SLING_MODEL_EXPORTER_NAME, extensions = ExporterConstants.SLING_MODEL_EXTENSION)
public class ImageDelegate implements Image {
    private static final Logger LOGGER = LoggerFactory.getLogger(ImageDelegate.class);
    public static final String RESOURCE_TYPE = "mysite/components/content/image";
    public static final String CONTENT_DAM_PATH = "/content/dam";
    public static final String HTTPS = "https://";
    public static final String PN_DISPLAY_SIZES = "sizes";
    public static final String WIDTH = "{.width}";

    @SlingObject
    private ResourceResolver resourceResolver;

    @SlingObject
    protected Resource resource;

    @Self
    @Via(type = ResourceSuperType.class)
    @Delegate(excludes = DelegationExclusion.class)
    private Image image;

    @Override
    public String getSrc() {
        return prepareSuffix(image.getSrc());
    }

    @Override
    public String getSrcset() {
        int[] widthsArray = image.getWidths();
        String srcUritemplate = image.getSrcUriTemplate();
        String[] srcsetArray = new String[widthsArray.length];
        if (widthsArray.length > 0 && srcUritemplate != null) {
            String srcUriTemplateDecoded = "";
            try {
                srcUriTemplateDecoded = prepareSuffix(URLDecoder.decode(srcUritemplate, StandardCharsets.UTF_8.name()));
            } catch (UnsupportedEncodingException e) {
                LOGGER.error("Character Decoding failed for {}", resource.getPath());
            }
            if (srcUriTemplateDecoded.contains(WIDTH)) {
                for (int i = 0; i < widthsArray.length; i++) {
                    if (srcUriTemplateDecoded.contains("="+WIDTH)) {
                        srcsetArray[i] = srcUriTemplateDecoded.replace(WIDTH, String.format("%s", widthsArray[i])) + " " + widthsArray[i] + "w";
                    } else {
                        srcsetArray[i] = srcUriTemplateDecoded.replace(WIDTH, String.format(".%s", widthsArray[i])) + " " + widthsArray[i] + "w";
                    }
                }
                return StringUtils.join(srcsetArray, ',');
            }
        }
        return null;
    }

    private String prepareSuffix(String imageSrc) {
        if(StringUtils.isNotEmpty(imageSrc) && !StringUtils.containsIgnoreCase(imageSrc, CONTENT_DAM_PATH)) {
            int endIndex = imageSrc.lastIndexOf(SlingPostConstants.DEFAULT_CREATE_SUFFIX);
            String intermittenResult = imageSrc.substring(0, endIndex);
            endIndex = intermittenResult.lastIndexOf(SlingPostConstants.DEFAULT_CREATE_SUFFIX);
            return intermittenResult.substring(0, endIndex)+image.getFileReference();
        }
        return imageSrc;
    }

    private interface DelegationExclusion {
        String getSrc();
        String getSrcset();
    }
}

In the above code, we are trying to prepend the URL with an image path, because we are using a synthetic image component for the content fragment

For more information on Lombok based component delegation please click on the title to check:

AEM Core Component Delegation

Create a new Adaptive Image servlet and EnhancedRendition class, this servlet is used for displaying appropriate rendition of image component based on width selected based on browser width and pixel ratio:

public class AdaptiveImageServlet  extends SlingSafeMethodsServlet {
    @Override
    protected void doGet(@NotNull SlingHttpServletRequest request, @NotNull SlingHttpServletResponse response) throws IOException {
        try {
            RequestPathInfo requestPathInfo = request.getRequestPathInfo();
            List<String> selectorList = selectorToList(requestPathInfo.getSelectorString());
            String suffix = requestPathInfo.getSuffix();
            String imagePath = suffix;
            String imageName = StringUtils.isNotEmpty(suffix) ? FilenameUtils.getName(suffix) : "";
            Resource component = request.getResource();
            ImageComponent imageComponent = new ImageComponent(component, imagePath);

        } catch (IllegalArgumentException e) {
            LOGGER.error("Invalid image request {}", e.getMessage());
            response.sendError(HttpServletResponse.SC_NOT_FOUND);
        }
    }
    private static class ImageComponent {
        Source source = Source.NONEXISTING;
        Resource imageResource;

        ImageComponent(@NotNull Resource component, String imagePath) {
            if (StringUtils.isNotEmpty(imagePath)) {
                imageResource = component.getResourceResolver().getResource(imagePath);
                source = Source.ASSET;
            }
        }
    }
}

Adaptive Image servlet is a modified version of the Core component Adaptive Image servlet because we are using synthetic image component and Enhanced Rendition class is a support class to get the best image rendition.

Create a simple content fragment Model as shown below:

Make sure the contentreference field name contains the image

Sample Content Fragment Model

Create a new content fragment under asset path (/content/dam/{project path}) as shown below:

Content Fragment

Create a sample page and add the content fragment component and select all the fields as a multifield as shown below:

Custom Content Fragment Component Authoing

As we can see we are able to fetch Core component Image component with Image widths configured into Image component policy (design dialog)

Content Fragment with OOTB Image

You can also get the actual working code from the below link:

https://github.com/kiransg89/ImageContentFragment

FAQ / Q&A using Content Fragments AEM

Problem Statement:

How can I get support for the Core Accordion component within the content fragment component?

Why should we use the Core Accordion component?

Requirement:

Get support for OOTB Accordion component within content fragment component

Introduction:

The Core Component Accordion component allows for the creation of a collection of components, composed as panels, and arranged in an accordion on a page, similar to the Tabs Component, but allows for expanding and collapsing of the panels.

  • The accordion’s properties can be defined in the configure dialog.
  • The order of the panels of the accordion can be defined in the configure dialog as well as the select panel popover.
  • Defaults for the Accordion Component when adding it to a page can be defined in the design dialog.

How to use Accordion for FAQ feature?

Manual authoring:

Drag and drop Accordion component, go to component dialog and click on add and select the component of your choice as shown below:

Accordion Dialog, insert panel

Once the component is selected add the question into the field

Author questions

Click on the panel selector icon and go to the individual item and make necessary changes

Panel Selection
Author answer

You can view it as published as shown below:

Published view of the Q&A

You can also rearrange the order of results from dialog

Dialog level reordering

Automated process:

Create a Sample model which can have a max of 10 Q&A’s as shown below:

Q&A content fragment

Create a custom Accordion component as shown below:

<div data-sly-use.accordion="com.adobe.cq.wcm.core.components.models.Accordion"
     data-sly-use.accordioncf="com.mysite.core.models.AccordionContentFragmentModel"
     data-panelcontainer="${wcmmode.edit && 'accordion'}"
     id="${accordion.id}"
     class="cmp-accordion"
     data-cmp-is="accordion"
     data-cmp-data-layer="${accordion.data.json}"
     data-cmp-single-expansion="${accordion.singleExpansion}"
     data-placeholder-text="${wcmmode.edit && 'Please drag Accordion item components here' @ i18n}">
    <h1>${accordioncf.faqTitle}</h1>
    <div data-sly-test="${accordion.items.size > 0}"
         data-sly-repeat.item="${accordion.items}"
         class="cmp-accordion__item"
         data-cmp-hook-accordion="item"
         data-cmp-data-layer="${item.data.json}"
         id="${item.id}"
         data-cmp-expanded="${item.name in accordion.expandedItems}">
        <h4 data-sly-element="${accordion.headingElement @ context='elementName'}"
            class="cmp-accordion__header">
            <button id="${item.id}-button"
                    class="cmp-accordion__button${item.name in accordion.expandedItems ? ' cmp-accordion__button--expanded' : ''}"
                    aria-controls="${item.id}-panel"
                    data-cmp-hook-accordion="button">
                <span class="cmp-accordion__title">${item.title}</span>
                <span class="cmp-accordion__icon"></span>
            </button>
        </h4>
        <div data-sly-resource="${item.name @ decorationTagName='div'}"
             data-cmp-hook-accordion="panel"
             id="${item.id}-panel"
             class="cmp-accordion__panel${item.name in accordion.expandedItems ? ' cmp-accordion__panel--expanded' : ' cmp-accordion__panel--hidden'}"
             role="region"
             aria-labelledby="${item.id}-button">
        </div>
    </div>
    <sly data-sly-resource="${resource.path @ resourceType='wcm/foundation/components/parsys/newpar', appendPath='/*', decorationTagName='div', cssClassName='new section aem-Grid-newComponent'}"
         data-sly-test="${(wcmmode.edit || wcmmode.preview) && accordion.items.size < 1}"></sly>
</div>

Create a Sling model interface called AccordionContentFragmentModel

package com.mysite.core.models;

import org.osgi.annotation.versioning.ConsumerType;

@ConsumerType
public interface AccordionContentFragmentModel {

    /**
     * Getter for checking editor template
     *
     * @return String content fragment FAQ Title
     */
    default String getFaqTitle() {
        throw new UnsupportedOperationException();
    }
}

Create a model implementation called as AccordionContentFragmentModelImpl

Make sure all the below code executes only on the author environment

We are checking for; how many content fragment fields are not empty and also we are checking for children’s under the current path.

If there is a difference, then we are going to delete the unwanted children’s and add the new children’s as shown below:

package com.mysite.core.models.impl;

import com.day.cq.commons.jcr.JcrConstants;
import com.day.cq.wcm.api.Page;
import com.mysite.core.models.AccordionContentFragmentModel;
import com.mysite.core.models.CFModel;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.math.NumberUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceUtil;
import org.apache.sling.models.annotations.Model;
import org.apache.sling.models.annotations.injectorspecific.OSGiService;
import org.apache.sling.models.annotations.injectorspecific.ScriptVariable;
import org.apache.sling.models.annotations.injectorspecific.Self;
import org.apache.sling.models.annotations.injectorspecific.SlingObject;
import org.apache.sling.models.factory.ModelFactory;
import org.apache.sling.settings.SlingSettingsService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.annotation.PostConstruct;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import java.util.stream.StreamSupport;

@Model(adaptables = SlingHttpServletRequest.class, adapters = {AccordionContentFragmentModel.class})
public class AccordionContentFragmentModelImpl implements AccordionContentFragmentModel {

    private static final Logger LOG = LoggerFactory.getLogger(AccordionContentFragmentModelImpl.class);

    private static final String CF_FRAGMENT_PATH = "fragmentPath";
    private static final String CF_ELEMENTS_NAME = "elementNames";
    private static final String CF_FAQ_QUESTION = "question";
    private static final String CF_FAQ_ANSWER = "answer";
    private static final String CF_FAQ_ITEMS = "/item_";
    private static final String CF_FAQ_PANEL_TITLE = "cq:panelTitle";

    private static final String CF_ARTICLE_FRAGMENT_PATH = "articleFragmentPath";
    private static final String CF_FAQ_ITEM = "item_";
    private static final String UNDERSCORE = "_";
    private static final String PUBLISH_ENV = "publish";

    private static final Map<Integer, String> NUMBERS = new HashMap<>();
    static {
        NUMBERS.put(1, "One");
        NUMBERS.put(2, "Two");
        NUMBERS.put(3, "Three");
        NUMBERS.put(4, "Four");
        NUMBERS.put(5, "Five");
        NUMBERS.put(6, "Six");
        NUMBERS.put(7, "Seven");
        NUMBERS.put(8, "Eight");
        NUMBERS.put(9, "Nine");
        NUMBERS.put(10, "Ten");
    }

    static Map<String, Object> defualtFaqProperties = new HashMap<>();
    static {
        defualtFaqProperties.put(JcrConstants.JCR_PRIMARYTYPE, JcrConstants.NT_UNSTRUCTURED);
        defualtFaqProperties.put(ResourceResolver.PROPERTY_RESOURCE_TYPE, "mysite/components/contentfragment");
        defualtFaqProperties.put("containerOpted", "dynamic");
        defualtFaqProperties.put("displayMode", "singleText");
        defualtFaqProperties.put("paragraphScope", "all");
        defualtFaqProperties.put("variationName", "master");
    }

    @Self
    private SlingHttpServletRequest request;

    @ScriptVariable
    private Resource resource;

    @ScriptVariable
    private Page currentPage;

    @SlingObject
    private ResourceResolver resourceResolver;

    @OSGiService
    private SlingSettingsService slingSettings;

    @OSGiService
    private ModelFactory modelFactory;

    private List<Integer> faqCount = new ArrayList<>();

    @PostConstruct
    private void initModel() {
        if(!isPublish(slingSettings)) {
            String pageArticleFragmentPath = currentPage.getProperties().get(CF_ARTICLE_FRAGMENT_PATH, StringUtils.EMPTY);
            if(StringUtils.isNotEmpty(pageArticleFragmentPath)) {
                Resource contentFragmentResource = resourceResolver.resolve(pageArticleFragmentPath);
                CFModel cfModel = modelFactory.getModelFromWrappedRequest(request, contentFragmentResource, CFModel.class);
                getBodyElementCounts(cfModel);
                List<Integer> childrens = StreamSupport.stream(resource.getChildren().spliterator(), false)
                        .map(accRes -> StringUtils.substringAfter(accRes.getName(), UNDERSCORE))
                        .map(NumberUtils::toInt)
                        .sorted()
                        .collect(Collectors.toList());
                if(!faqCount.equals(childrens)) {
                    deleteInvalidResource(childrens);
                    addFaqItem(faqCount, resourceResolver, resource, pageArticleFragmentPath, cfModel);
                }
            }
        }
    }

    private void deleteInvalidResource(List<Integer> childrens) {
        childrens.removeAll(faqCount);
        if(!childrens.isEmpty()) {
            childrens.stream().forEach(this::deleteResource);
            commitChanges();
        }
    }
    private void commitChanges() {
        if(resourceResolver.hasChanges()) {
            try {
                resourceResolver.commit();
            } catch (PersistenceException e) {
                LOG.error("Unable save the cahnges {}",e.getMessage());
            }
        }
    }

    private void deleteResource(Integer num) {
        try {
            resourceResolver.delete(resource.getChild(CF_FAQ_ITEM+num));
        } catch (PersistenceException e) {
            LOG.error("Unable to delete the resource {}", e.getMessage());
        }
    }

    public void getBodyElementCounts(CFModel cfModel) {
        faqCount = IntStream.rangeClosed(1, 20).filter(num -> checkFragment(num, cfModel, CF_FAQ_QUESTION)).boxed().collect(Collectors.toList());
    }

    public static boolean checkFragment(int num, CFModel cfModel, String elementName) {
        return StringUtils.isNotEmpty(cfModel.getElementContent(elementName+ NUMBERS.get(num)));
    }

    public static void addFaqItem(List<Integer> faqCount, ResourceResolver resourceResolver, Resource resource, String pageArticleFragmentPath, CFModel cfModel) {
        while(!faqCount.isEmpty()) {
            try {
                Map<String, Object> faqProperties = defualtFaqProperties;
                faqProperties.put(CF_FRAGMENT_PATH, pageArticleFragmentPath);
                faqProperties.put(CF_ELEMENTS_NAME, CF_FAQ_ANSWER+ NUMBERS.get(faqCount.get(0)));
                faqProperties.put(CF_FAQ_PANEL_TITLE, cfModel.getElementContent(CF_FAQ_QUESTION+ NUMBERS.get(faqCount.get(0))));
                ResourceUtil.getOrCreateResource(resourceResolver, resource.getPath()+CF_FAQ_ITEMS+faqCount.get(0), faqProperties, StringUtils.EMPTY, false);
                faqCount.remove(0);
            } catch (PersistenceException e) {
                LOG.error("Unable to persist the resource {}", e.getMessage());
            }
        }
    }

    public static boolean isPublish(SlingSettingsService slingSettings) {
        return slingSettings.getRunModes().contains(PUBLISH_ENV);
    }
}

You can author the component by selecting the fragment path.

Notes: Please make sure we don’t author anything manually by clicking on add item

Author content fragment path

After authoring you can reload the page to see the content is being populated into the accordion model and all the changes made on the content fragment will be pulled into the component

Content fragment data is pulled automatically

You can also rearrange the order of the FAQ from dialog level or from the panel selector.

Rearrange from panel selector

Flexibility Vs Automation:

This component will automatically pull Q&A from content fragment but if there is any update on the question field then it won’t reflect hence please remove the field which is not pulling the latest results and refresh the page so that it pulls the latest.

Why are we seeing above issue?

We are unable to pull the updated questions into the component because this will remove the flexibility of rearranging the panels. Based on your requirement you can remove this functionality.

you can also find the working code from here:

https://github.com/kiransg89/AccordionContentFragment