Broken Page References AEM


Problem Statement:

How to get the list of all the broken references in AEM?


Requirement:

Get a List of all the broken references using MCP and provide the report


Introduction:

OOTB we get a Broken reference report provided by MCP, which can be used to get all the broken references in the content repo.

Broken Refernce Report

It’s highly recommended to run this process during

  1. off hours
  2. Don’t run on the root level
  3. Run it on 2nd level or 3rd level pages

How to run this process?

Provide Source path

Provide the regex so that it will consider only the references which point to /content or /etc (points to AEM)

You can also provide exclude properties to improve the traversal of nodes.

If you want to verify any broken links in the RTE fields or properties, then check the deep check checkbox and provide the properties list.

But the above process has a few issues.

  1. Html properties are not working as expected

We need a few customizations to this process by making a few changes to check HTML level references by adding JSOUP API

Add the following dependencies to your POM.xml

<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>com.adobe.acs</groupId>
    <artifactId>acs-aem-commons-bundle</artifactId>
    <scope>provided</scope>
</dependency>

Get the following Broken reference code into your local as shown below:

Add the following code as shown below:

if (htmlFields.contains(property.getKey())) {
            stream = stream.flatMap(val -> {
                try {
                    Document doc = Jsoup.parse(val);
                    Elements anchors = doc.select("a");
                    return anchors.stream().map(link -> link.attr("href"));
                } catch (Exception e) {
                    log.warn("Could not parse links from property value of {}", property.getKey(), e);
                    return Stream.empty();
                }
            });
        }
At Line number 207

When we run it on wknd site it would look something like this:

Broken Reference Report

Broken Asset References AEM


Problem Statement:

Get a List of all the Assets which are missing references


Requirement:

Get the list of broken asset references to unpublish and remove them repo to improve the system stability and performance.


Introduction:

How do assets get published?

  1.  The author uploads the images and publishes the assets
  2. Create a launcher and workflow which process assets metadata and publish the pages
  3. Whenever we publish any pages and if the page has references to assets, then during publishing, it asks to replicate the references as well.

What happens when the page is unpublished?

  1.  When the page is deactivated, assets referenced to the page will not be deactivated because this asset might have reference to the other pages hence out of the box assets won’t be deactivated.
  2. If we perform cleanup, deactivate and delete old pages, we might not be cleaning up assets related to this page.

Advantages of cleaning up old assets?

  1. Drastically reduces repository size
  2. Improves DAM Asset search
  3. Improves indexing

Get Publish Report using Assets Report:

  • Go to Tools -> Assets -> Reports as shown below:
Asset Reports
  • Click on create and click on Publish report
Select Publish report
  • Provide folder path and start date and end date
Add Report details
  • Select the columns as per requirement
Configure columns for the report
  • Finally, report will be ready with all the assets lists as shown below
Completed Reports
  • Download the report to see the final list of images
Example Report CSV file

If Images are unpublished then we can ask authors to review and delete them

If images are published but has no references to figure this out, we need a new process.

Broken Asset reference:

Add the following dependencies to your pom.xml

        <dependency>
            <groupId>org.jetbrains</groupId>
            <artifactId>annotations</artifactId>
            <version>17.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.adobe.acs</groupId>
            <artifactId>acs-aem-commons-bundle</artifactId>
            <version>5.2.0</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.jsoup</groupId>
            <artifactId>jsoup</artifactId>
            <version>1.13.1</version>
            <scope>provided</scope>
        </dependency>

Create a new MCP process by calling the MCP service and providing the implementation class:

package com.mysite.core.mcp;

import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import com.adobe.acs.commons.mcp.ProcessDefinitionFactory;
import com.day.cq.replication.Replicator;

@Component(service = ProcessDefinitionFactory.class)
public class BrokenAssetsFactory extends ProcessDefinitionFactory<BrokenAssets> {

    @Reference
    Replicator replicator;

    @Override
    public String getName() {
        return "Broken Asset References";
    }

    @Override
    public BrokenAssets createProcessDefinitionInstance() {
        return new BrokenAssets(replicator);
    }
}

Create an implementation class as shown below:

package com.mysite.core.mcp;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.EnumMap;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;
import javax.jcr.RepositoryException;
import org.apache.commons.collections4.ListUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.jackrabbit.util.Text;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.LoginException;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.jetbrains.annotations.NotNull;
import com.adobe.acs.commons.data.CompositeVariant;
import com.adobe.acs.commons.data.Spreadsheet;
import com.adobe.acs.commons.fam.ActionManager;
import com.adobe.acs.commons.mcp.ProcessDefinition;
import com.adobe.acs.commons.mcp.ProcessInstance;
import com.adobe.acs.commons.mcp.form.Description;
import com.adobe.acs.commons.mcp.form.FileUploadComponent;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.RadioComponent;
import com.adobe.acs.commons.mcp.model.GenericReport;
import com.adobe.acs.commons.mcp.model.ManagedProcess;
import com.day.cq.replication.ReplicationStatus;
import com.day.cq.replication.Replicator;

/**
 * Relocate Pages and/or Sites using a parallelized move process
 */
public class BrokenAssets extends ProcessDefinition {
	
	private static final String SOURCE_PATH = "Path";
	private static final String CONTAINS_QUERY = "CONTAINS(s.*, '%s') or ";
	private final GenericReport report = new GenericReport();
	List<EnumMap<ReportColumns, String>> reportData = Collections.synchronizedList(new ArrayList<>());
	
	List<String> assetsList = new LinkedList<>();
	
    public enum PublishMethod {
        @Description("Select this option to generate Broken References Report")
        BROKEN_REFERENCE_REPORT
    }
    
    Replicator replicatorService;
    
    @FormField(name = "Mapping Excel", component = FileUploadComponent.class)
    private RequestParameter mappingExcel;
    private Spreadsheet spreadsheet;
    
    @FormField(name = "Type",
            description = "Please select the oprporiate option to execute",
            component = RadioComponent.EnumerationSelector.class,
            required = true,
            options = {"vertical", "default=BROKEN_REFERENCE_REPORT"})
    public PublishMethod publishMethod = PublishMethod.BROKEN_REFERENCE_REPORT;

	@FormField(name="Content Path",
			description="Content Path for search results",
			hint="/content",
			options={"default=/content"})
	public String contentPath = "/content";
    
    @FormField(name="Chunk count",
            description="Max number of chunk the search results",
            hint="4500",
            options={"default=3000"})
    public int chunkCount = 3000;
    
    @FormField(name="Retries",
            description="Max number of retries per commit",
            hint="2",
            options={"default=2"})
    public int retryCount = 2;
    
    @FormField(name="Retry delay",
            description="Delay between retries (in milliseconds)",
            hint="500,1000,...",
            options={"default=500"})
    public int retryWait = 500;
    
    public BrokenAssets(Replicator replicator) {
    	replicatorService = replicator;
	}

    @Override
    public void init() throws RepositoryException {    	
    	if(null != mappingExcel && mappingExcel.getSize() > 0) {
    		try {
                // Read spreadsheet
        		spreadsheet = new Spreadsheet(mappingExcel, SOURCE_PATH).buildSpreadsheet();
        		spreadsheet.getDataRowsAsCompositeVariants().forEach(row -> assetsList.add(getString(row, SOURCE_PATH)) );
            } catch (IOException e) {
                throw new RepositoryException("Unable to process spreadsheet", e);
            }
    	}    	    	    	
    }

    ManagedProcess instanceInfo;

    @Override
    public void buildProcess(ProcessInstance instance, ResourceResolver rr) throws LoginException, RepositoryException {
        instanceInfo = instance.getInfo();
        switch (publishMethod.name().toLowerCase()) {
	        case "broken_reference_report":
	        	instance.getInfo().setDescription("Collecting references");
	        	instance.defineAction("Searching Refs", rr, this::collectRefernce);
	            break;
        }        
    }

    @Override
    public void storeReport(ProcessInstance instance, ResourceResolver rr) throws RepositoryException, PersistenceException {    	
        report.setRows(reportData, ReportColumns.class);
        report.persist(rr, instance.getPath() + "/jcr:content/report");
    }

    public enum ReportColumns {
    	ASSET_PATH
    }
            
    protected void collectRefernce(ActionManager manager) {
    	manager.deferredWithResolver(this::collectReferences); 
    }
    
    private void collectReferences(ResourceResolver resourceResolver) throws Exception {    	
		if(!assetsList.isEmpty()) {						 
			RetryUtils.withRetry(retryCount, retryWait, () -> {
			    List<List<String>> chunkedAssetList = ListUtils.partition(assetsList, chunkCount);					
			    int counter = 0;
			    while(!chunkedAssetList.isEmpty() && counter < chunkedAssetList.size()) {
			        List<String> chunk = chunkedAssetList.get(counter);
			        counter++;
			        @NotNull Iterator<Resource> resourceResults = buildSQLQueryAndFetchResults(resourceResolver, contentPath, chunk);
			        collectReferences(resourceResults, chunk);
			    }
			});	
			if(!assetsList.isEmpty()) {
				assetsList.stream().forEach(this::reportResult);
			}
		}	        			
    }

	private void collectReferences(@NotNull Iterator<Resource> resourceResults, List<String> chunk) {
		while(resourceResults.hasNext()) {
			Resource resultRes = resourceResults.next();
			if (null != resultRes) {
				resultRes.getValueMap().entrySet().forEach(property -> {
					if(!property.getKey().equalsIgnoreCase("dam:folderThumbnailPaths")) {
						Object prop = property.getValue();
						if (prop.getClass() == String[].class) {
							List<String> propertyValue = Arrays.asList((String[]) prop);
							if (!propertyValue.isEmpty()) {
								List<String> matchingAsset = chunk.stream().filter(
										assetPat -> propertyValue.stream().anyMatch(sam -> sam.contains(assetPat)))
										.collect(Collectors.toList());
								if(!matchingAsset.isEmpty()) {
									chunk.removeIf(matchingAsset::contains);
									assetsList.removeIf(matchingAsset::contains);
								}
							}
						} else if (prop.getClass() == String.class) {
							String propertyValue = (String) prop;
							if (StringUtils.isNotEmpty(propertyValue) ) {
								Optional<String> matchingAsset = chunk.stream().filter(propertyValue::contains).findAny();
								if(matchingAsset.isPresent()) {
									chunk.remove(matchingAsset.get());
									assetsList.remove(matchingAsset.get());
								}
							}
						}
					}
				});
			}
		}
	}

	private void reportResult(String path) {
		EnumMap<ReportColumns, String> row = new EnumMap<>(ReportColumns.class);
		row.put(ReportColumns.ASSET_PATH, path);		
		reportData.add(row);
	}	
	
	private @NotNull Iterator<Resource> buildSQLQueryAndFetchResults(ResourceResolver resolver, String contentPath, List<String> chunk) {
        String groupStr = chunk.stream().map(r -> String.format(CONTAINS_QUERY, Text.escapeIllegalXpathSearchChars(r).replaceAll("'", "''"))).collect(Collectors.joining());        
		String querySt = "SELECT * FROM [nt:base] AS s WHERE ISDESCENDANTNODE(["+contentPath+"]) and "+ StringUtils.substringBeforeLast(groupStr, "or");
		return resolver.findResources(querySt, "JCR-SQL2");
    }
	
	@SuppressWarnings({"rawtypes", "unchecked"})
    private String getString(Map<String, CompositeVariant> row, String path) {
        CompositeVariant v = row.get(path.toLowerCase(Locale.ENGLISH));
        if (v != null) {
            return (String) v.getValueAs(String.class);
        } else {
            return null;
        }
    }
}
package com.mysite.core.mcp;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RetryUtils {
    private static final Logger log = LoggerFactory.getLogger(RetryUtils.class);

    public static interface CallToRetry {
        void process() throws Exception;
    }

    public static boolean withRetry(int maxTimes, long intervalWait, CallToRetry call) throws Exception {
        if (maxTimes <= 0) {
            throw new IllegalArgumentException("Must run at least one time");
        }
        if (intervalWait <= 0) {
            throw new IllegalArgumentException("Initial wait must be at least 1");
        }
        Exception thrown = null;
        for (int i = 0; i < maxTimes; i++) {
            try {
                call.process();
                return true;
            } catch (Exception e) {
                thrown = e;
                log.info("Encountered failure on {} due to {}, attempt retry {} of {}", call.getClass().getName() , e.getMessage(), (i + 1), maxTimes, e);
            }
            try {
                Thread.sleep(intervalWait);
            } catch (InterruptedException wakeAndAbort) {
                break;
            }
        }
        throw thrown;
    }
}

Running Asset Reference Process:

After building the code you can see the new Process showing up in MCP

Borken Asset Refernce Process

Copy the Path column into the new Excel sheet as shown below

Path column into new excel file

Upload into the process and start to see all the images which are published yet unreferenced as shown below

Why does Chunk count?

Chunk count helps the SQL 2 query to group by the paths, which will be maxing 4500 and it won’t take more than that (configurable based on the environment). However basically, if we have 20000 / 4500 = 4.44 ~ 5 we will be running the query max five times to generate the below report

Share the report with the content authors team to validate if images are required if not plan to clean up using

After completing the process
Example report and we can download and share with Authors

Clean up Process:

Authors don’t have to unpublish and delete individual images, then can you use the below process to upload the excel sheet with all the approved image paths and upload it to the process to deactivate and delete

AEM Publish / UnPublish / Delete List of pages – MCP Process

Unclosed resource resolver complete guide AEM


Problem Statement:

Unclosed resource resolver issue is causing performance impact on the AEM environment

It’s hard to debug and fix the resolver issues

Best practices to close the resolver


Requirement:

Provide all the best practices to close the unclosed resource resolve and in turn improve environment stability.


Introduction:

Usually, we create/open a resource resolver:

  1. Servlets
  2. OSGi Services
  3. Workflows
  4. Schedulers

Purpose of service user based resolver.

To get access to the paths which is blocked for every one group, make changes and commit changes etc.

Creating Resource resolver instance:

This feature was released with Java 7 and the try-with-resources statement is a try statement that declares one or more resources. A resource is an object that must be closed after the program is finished with it. The try-with-resources statement ensures that each resource is closed at the end of the statement. Any object that implements java.lang.AutoCloseable, which includes all objects which implement java.io.Closeable, can be used as a resource.

So basically, creating the resolver within the calling class like below will auto close:

try (ResourceResolver resourceResolver = resolverFactory.getServiceResourceResolver(Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "{service user name}"))){
    //business logic
}

And it is highly recommended to use the following APIs as well:

  1. FileReader
  2. ZipFile
  3. BufferedWriter

etc.

Closing the resolver for Queries:

Whenever we come across queries, we need to close the resolver manually even after using the try resource approach:

Try resource Approach:

ResourceResolver resolverResolverLeakingReference = null;
try (ResourceResolver resolver = resourceResolverFactory.getServiceResourceResolver(Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "{service user name}"))) {
    session = resolver.adaptTo(Session.class);
    queryBuilder = resolver.adaptTo(QueryBuilder.class);
    
    PredicateGroup predicateGroup = null;
    try {
        predicateGroup = PredicateGroup.create(map);
    } catch (VerifyError ex) {
        predicateGroup = new PredicateGroup();
    }

    Query query = queryBuilder.createQuery(predicateGroup, session);
    SearchResult result = query.getResult();
    for (final Hit hit : result.getHits()) {
        Resource resource = hit.getResource();            	
        if(resolverResolverLeakingReference == null){
            resolverResolverLeakingReference =  resource.getResourceResolver();
        }    
        //buisness logic                            
    }
} catch (Exception e) {
    log.error("Error fetching locations count results", e);
} finally {
    if (resolverResolverLeakingReference != null) {
        // Always Close the leaking QueryBuilder resourceResolver.
        resolverResolverLeakingReference.close();    
    }        
}

Requet.getResolver Approach:

ResourceResolver leakingResourceResolverReference = null;
try {
    Session session = request.getResourceResolver().adaptTo(Session.class);
    Map<String, String> map = new HashMap<>();
    //map put goes here
        Query query = queryBuilder.createQuery(PredicateGroup.create(map), session);
        SearchResult result = query.getResult();
        for (Hit hit : result.getHits()) {
            if(leakingResourceResolverReference == null) {
                leakingResourceResolverReference =  hit.getResource().getResourceResolver();
            }		         
            //buisness logic
        }    
} catch(Exception e) {
    log.info("error::{}",e.getMessage());
} finally {
    if(leakingResourceResolverReference != null){
        leakingResourceResolverReference.close();
    }
}

Or you can also follow this:

SearchResult result;
try {
    Session session = request.getResourceResolver().adaptTo(Session.class);
    Map<String, String> map = new HashMap<>();
    //map put goes here
        Query query = queryBuilder.createQuery(PredicateGroup.create(map), session);
        result = query.getResult();
        for (Hit hit : result.getHits()) {    
            //buisness logic
        }    
} catch(Exception e) {
    log.info("error::{}",e.getMessage());
} finally {
    Iterator<Resource> resources = result.getResources();
    if (resources.hasNext()) {
        resources.next().getResourceResolver().close();
    }
}

Avoid query builder and use SQL2:

try (ResourceResolver resourceResolver = resolverFactory.getServiceResourceResolver(Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "{service user name}"))){
    String querySt = "SELECT * FROM [nt:base] ";		
    Iterator<Resource> results = resourceResolver.findResources(querySt, "JCR-SQL2");
    //buisness logc
}

Use Java Streams:

You can follow the below blog to use Java stream for executing queries in AEM

AEM Query builder using Java streams

Purge Old CRX Packages AEM


Problem statement:

AEM environment size is increasing because of user-generated packages


Requirement:

Can we purge all the user-generated packages to improve stability?


Introduction:

A package is a zip file holding repository content in the form of a file-system serialization (called “vault” serialization). This provides an easy-to-use-and-edit representation of files and folders.

Packages include content, both page content and project-related content, selected using filters.

A package also contains vault meta information, including the filter definitions and import configuration information. Additional content properties (that are not used for package extraction) can be included in the package, such as a description, a visual image, or an icon; these properties are for the content package consumer and for informational purposes only.

Usually, Developers or AEM content synch or even code deployment will keep on piling up in the CRX packages and it will be consuming spaces on MBs and even sometimes GBs.

If we move more packages, then loading crx/packmgr would more time.

Hence you can create a scheduler that runs on off-hours which cleans up the packages and which will get back the space and avoids extra maintenance tasks.

The below scheduler will remove all the packages for my_package group we can add business logic to handle for other groups

package com.mysite.core.schedulers;

import org.osgi.service.metatype.annotations.AttributeDefinition;
import org.osgi.service.metatype.annotations.AttributeType;
import org.osgi.service.metatype.annotations.ObjectClassDefinition;

@ObjectClassDefinition(name = "Old Packages Purge Schedular", description = "Remove old packages from different paths")
public @interface PurgeOldPackagesSchedulerConfig {

    String DEFAULT_SCHEDULER_EXPRESSION = "0 0 16 ? * SUN *"; // every Sunday 4 PM
    boolean DEFAULT_SCHEDULER_CONCURRENT = false;

    @AttributeDefinition(name = "Enabled", description = "True, if scheduler service is enabled", type = AttributeType.BOOLEAN)
    boolean enabled() default true;

    @AttributeDefinition(name = "Cron expression defining when this Scheduled Service will run", description = "[every minute = 0 * * * * ?], [12:00am daily = 0 0 0 ? * *]", type = AttributeType.STRING)
    String schedulerExpression() default DEFAULT_SCHEDULER_EXPRESSION;

    @AttributeDefinition(name = "package paths", description = "package folder paths", type = AttributeType.STRING)
    String[] packagesPaths() default {"my_packages"};
}
package com.mysite.core.schedulers;

import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import org.apache.jackrabbit.vault.packaging.JcrPackage;
import org.apache.jackrabbit.vault.packaging.JcrPackageManager;
import org.apache.jackrabbit.vault.packaging.Packaging;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceResolverFactory;
import org.apache.sling.commons.scheduler.ScheduleOptions;
import org.apache.sling.commons.scheduler.Scheduler;
import org.osgi.service.component.annotations.Activate;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.ConfigurationPolicy;
import org.osgi.service.component.annotations.Deactivate;
import org.osgi.service.component.annotations.Modified;
import org.osgi.service.component.annotations.Reference;
import org.osgi.service.metatype.annotations.Designate;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Component(immediate = true, service = PurgeOldPackagesScheduler.class, configurationPolicy = ConfigurationPolicy.REQUIRE)
@Designate(ocd = PurgeOldPackagesSchedulerConfig.class)
public class PurgeOldPackagesScheduler implements Runnable {

    private static final Logger log = LoggerFactory.getLogger(PurgeOldPackagesScheduler.class);
    /**
     * Id of the scheduler based on its name
     */
    private String schedulerJobName;

    Session session;
    private List<String> packagesPathsList;

    SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");

    @Reference
    private Packaging packaging;

    @Reference
    private Scheduler scheduler;

    @Reference
    private ResourceResolverFactory resolverFactory;

    @Activate
    @Modified
    protected void activate(PurgeOldPackagesSchedulerConfig purgeOldPackagesSchedulerConfig) {
        /**
         * Creating the scheduler id
         */

        this.schedulerJobName = this.getClass().getSimpleName();
        addScheduler(purgeOldPackagesSchedulerConfig);
        packagesPathsList = Arrays.asList(purgeOldPackagesSchedulerConfig.packagesPaths());
    }

    /**
     * @see Runnable#run().
     */
    @Override
    public final void run() {
        log.debug("PurgeOldPackagesScheduler Job started");
        purgeOldPackages(packagesPathsList);
        log.debug("PurgeOldPackagesScheduler Job completed");
    }

    void purgeOldPackages(List<String> packagesPathsList) {
        try (ResourceResolver resourceResolver = resolverFactory.getServiceResourceResolver(Collections
                .singletonMap(ResourceResolverFactory.SUBSERVICE, ServiceUserConstants.ADMINISTRATIVE_SERVICE_USER))) {
            session = resourceResolver.adaptTo(Session.class);
            JcrPackageManager jcrPackageManager = packaging.getPackageManager(session);
            packagesPathsList.forEach(group -> {
                removePackages(jcrPackageManager, 0, group);
            });
        } catch (Exception e) {
            log.error(e.getMessage(), e);
        } finally {
            if (session != null) {
                session.logout();
            }
        }
    }

    @Deactivate
    protected void deactivate() {

        /**
         * Removing the scheduler
         */
        removeScheduler();
    }

    /**
     * This method adds the scheduler
     *
     * @param purgeOldPackagesSchedulerConfig
     */
    private void addScheduler(PurgeOldPackagesSchedulerConfig purgeOldPackagesSchedulerConfig) {

        /**
         * Check if the scheduler is enabled
         */
        if (purgeOldPackagesSchedulerConfig.enabled()) {

            /**
             * Scheduler option takes the cron expression as a parameter and run accordingly
             */
            ScheduleOptions scheduleOptions = scheduler.EXPR(purgeOldPackagesSchedulerConfig.schedulerExpression());

            /**
             * Adding some parameters
             */
            scheduleOptions.name(schedulerJobName);
            scheduleOptions.canRunConcurrently(false);

            /**
             * Scheduling the job
             */
            scheduler.schedule(this, scheduleOptions);

            log.info("{} Scheduler added", schedulerJobName);
        } else {
            log.info("Scheduler is disabled");
            removeScheduler();
        }

    }

    /**
     * This method removes the scheduler
     */
    private void removeScheduler() {

        log.info("Removing scheduler: {}", schedulerJobName);

        /**
         * Unscheduling/removing the scheduler
         */
        scheduler.unschedule(String.valueOf(schedulerJobName));
    }

    private void removePackages(JcrPackageManager jcrPackageManager, int counterStart, String groupName) {
        try {
            List<JcrPackage> packages = jcrPackageManager.listPackages(groupName, false);
            AtomicInteger counter = new AtomicInteger(counterStart);
            if (null != packages && !packages.isEmpty()) {
                packages.stream().sorted(Comparator.nullsLast((e1, e2) -> {
                    try {
                        return ((JcrPackage) e1).getPackage().getCreated()
                                .compareTo(((JcrPackage) e2).getPackage().getCreated());
                    } catch (RepositoryException | IOException ex) {
                        log.error(ex.getMessage());
                    }
                    return -1;
                }).reversed()).forEach(pack -> {
                    if (counter.incrementAndGet() > 3) {
                        try {
                            jcrPackageManager.remove(pack);
                        } catch (RepositoryException ex) {
                            log.error(ex.getMessage());
                        }
                    }
                });
            }
        } catch (RepositoryException e) {
            e.printStackTrace();
        }
    }
}

Caching AEM GraphQL queries with content fragment


Problem Statement:

How can I persist query?

How to cache my query results?
How to Update my queries?


Requirement:

Provide details on how to add the persist graphql query, cache the results from graphql and update the persisted query

Provide curl commands to execute in terminal or on postman


Introduction:

Persisted Queries (Caching)

After preparing a query with a POST request, it can be executed with a GET request that can be cached by HTTP caches or a CDN.

This is required as POST queries are usually not cached, and if using GET with the query as a parameter there is a significant risk of the parameter becoming too large for HTTP services and intermediates.

Persisted queries must always use the endpoint related to the appropriate Sites configuration; so, they can use either, or both:

  • Specific Sites configuration and endpoint

Creating a persisted query for a specific Sites configuration requires a corresponding Sites-configuration-specific endpoint (to provide access to the related Content Fragment Models).

For example, to create a persisted query specifically for the SampleGraphQL Sites configuration:

a corresponding SampleGraphQL-specific Sites configuration

  • Go to the tools section for the aem and general section and select Configuration Browser as shown below
Configuration browser
  • Add select the conf folder and go to the properties and make GraphQL Persistent Queries checkbox is checked
Enable persistent queries

a SampleGraphQL-specific endpoint must be created in advance.

  • Go to tools section for the aem and assets section and select GraphQL as shown below
assets -> graphql
  • Add the new end point as shown below:
endpoint

Add the following CORS configurations for the GraphQL API calls:

CORS config

Register graphql search path:

Register Servlet path

Here are the steps required to persist a given query:

Prepare the query by putting it to the new endpoint URL /graphql/persist.json/<config>/<persisted-label>.

For example, create a persisted query:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/cities' \
--header 'Content-Type: application/json' \
--data-raw '{
  cityList {
    items {
      _path
      name
      country
      population
    }
  }
}'
  • At this point, check the response.

For example, check for success:

{
    "action": "create",
    "configurationName": "SampleGraphQL",
    "name": "cities",
    "shortPath": "/SampleGraphQL/cities",
    "path": "/conf/SampleGraphQL/settings/graphql/persistentQueries/cities"
}

You can then replay the persisted query by getting the URL /graphql/execute.json/<shortPath>.

For example, use the persisted query:

curl -u admin:admin -X GET 'http://localhost:4502/graphql/execute.json/SampleGraphQL/cities' \
--header 'Authorization: Basic YWRtaW46YWRtaW4='

Update a persisted query by POSTing to an already existing query path.

For example, use the persisted query:

curl -u admin:admin -X POST 'http://localhost:4502/graphql/persist.json/SampleGraphQL/cities' \
--header 'Content-Type: application/json' \
--data-raw '{
  cityList {
    items {
      _path
      name
      country
      population
    }
  }
}'

Create a wrapped plain query.

For example:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/plain-cities' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "{cityList { items { _path name country country population } } }"}'

Create a wrapped plain query with cache control.

For example:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/plain-cities-max-age' \
--header 'Content-Type: application/json' \
--data-raw '{ "query": "{cityList { items { _path name country country population } } }", "cache-control": { "max-age": 300 }}'

Create a persisted query with parameters:

For example:

curl -u admin:admin -X PUT 'http://localhost:4502/graphql/persist.json/SampleGraphQL/plain-cities-query-parameters' \
--header 'Content-Type: application/json' \
--data-raw \
'query GetAsGraphqlModelTestByPath($apath: String!) {
    cityByPath(_path: $apath) {
        item {
        _path
        name
        country
        population
        }
    }
  }'

Executing a query with parameters.

For example:

curl -u admin:admin -X POST \
    -H "Content-Type: application/json" \
    "http://localhost:4502/graphql/execute.json/SampleGraphQL/plain-cities-query-parameters;apath=%2Fcontent%2Fdam%2Fsample-content-fragments%2Fcities%2Fberlin"

curl -u admin:admin -X GET \
    "http://localhost:4502/graphql/execute.json/SampleGraphQL/plain-cities-query-parameters;apath=%2Fcontent%2Fdam%2Fsample-content-fragments%2Fcities%2Fberlin"

Bulk Add, Update and Delete properties in AEM – without using Groovy console

Problem Statement:

How to Bulk Add, Update or remove page properties in AEM? Without using the Groovy console.

Requirement:

Create a reusable process that can be used to search for the pages based on resourceType and do the CRUD operations on the results.

Introduction:

Usually, whenever we are using editable templates, we might have some initial content but for some reason, if we want to update the experience fragment path or some page properties then usually, we go for Groovy script to run bulk update.

But AMS don’t install developer tools on the PROD, we need to go to other options and for the above requirement, we can use MCP.

MCP (Manage Controlled Processes) is both a dashboard for performing complex tasks and a rich API for defining these tasks as process definitions. In addition to kicking off new processes, users can also monitor running tasks, retrieve information about completed tasks, halt work, and so on.

Add the following maven dependency to your pom to extend MCP

<dependency>
    <groupId>com.adobe.acs</groupId>
    <artifactId>acs-aem-commons-bundle</artifactId>
    <version>5.0.4</version>
    <scope>provided</scope>
</dependency>

Create Process Definition factory – PropertyUpdateFactory

This class tells ACS Commons MCP to pick the process definition and process name getName and you need to mention the implementation class inside the createProcessDefinitionInstance method as shown below:

package com.mysite.mcp.process;

import org.osgi.service.component.annotations.Component;
import com.adobe.acs.commons.mcp.ProcessDefinitionFactory;

@Component(service = ProcessDefinitionFactory.class, immediate = true)
public class PropertyUpdateFactory extends ProcessDefinitionFactory<PropertyUpdater> {
    @Override
    public String getName() {
        return "Property Updator";
    }
    @Override
    protected PropertyUpdater createProcessDefinitionInstance() {
        return new PropertyUpdater();
    }
}

Create Process Definition implementation – PropertyUpdater

This is an implementation class where we are defining all the form fields required for the process to run

package com.mysite.mcp.process;

import java.io.IOException;
import java.text.MessageFormat;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Collections;
import java.util.EnumMap;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import javax.jcr.RepositoryException;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.request.RequestParameter;
import org.apache.sling.api.resource.LoginException;
import org.apache.sling.api.resource.ModifiableValueMap;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceUtil;
import org.apache.sling.resource.filter.ResourceFilterStream;
import org.jetbrains.annotations.NotNull;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.adobe.acs.commons.data.Spreadsheet;
import com.adobe.acs.commons.fam.ActionManager;
import com.adobe.acs.commons.mcp.ProcessDefinition;
import com.adobe.acs.commons.mcp.ProcessInstance;
import com.adobe.acs.commons.mcp.form.FileUploadComponent;
import com.adobe.acs.commons.mcp.form.FormField;
import com.adobe.acs.commons.mcp.form.SelectComponent;
import com.adobe.acs.commons.mcp.form.TextfieldComponent;
import com.adobe.acs.commons.mcp.model.GenericReport;
import com.day.cq.commons.jcr.JcrConstants;

public class PropertyUpdater extends ProcessDefinition {

    private static final Logger LOGGER = LoggerFactory.getLogger(PropertyUpdater.class);
    private final GenericReport report = new GenericReport();
    private static final String REPORT_NAME = "Property-update-report";
    private static final String RUNNING = "Running ";
    private static final String EXECUTING_KEYWORD = " Property Updation";

    private static final String PROPERTY = "property";
    private static final String PROPERTY_TYPE = "type";
    private static final String PROPERTY_VALUE = "value";

    protected enum UpdateAction {
        ADD, UPDATE, DELETE
    }

    @FormField(name = "Property Update Excel", component = FileUploadComponent.class)
    private RequestParameter sourceFile;

    @FormField(name = "Path",
            component = TextfieldComponent.class,
            hint = "(Provide the Relative path to start search)",
            description = "A query will be executed starting this path")
    private String path;

    @FormField(name = "Action",
            component = SelectComponent.EnumerationSelector.class,
            description = "Add, Update or Delete?",
            options = "default=Add")
    UpdateAction reAction = UpdateAction.ADD;

    @FormField(name = "ResourceType",
            component = TextfieldComponent.class,
            hint = "(Provide the resourcetype to be searched for)",
            description = "A query will be executed based on resourcetype")
    private String pageResourceType;

    private Map<String, Object> propertyMap = new HashMap<>();

    @Override
    public void init() throws RepositoryException {
        validateInputs();
    }

    @Override
    public void buildProcess(ProcessInstance instance, ResourceResolver rr) throws LoginException, RepositoryException {
        report.setName(REPORT_NAME);
        instance.getInfo().setDescription(RUNNING + reAction + EXECUTING_KEYWORD);
        instance.defineCriticalAction("Adding the Properties", rr, this::updateProperties);
    }

    private void updateProperties(ActionManager manager) {
        manager.deferredWithResolver(this::addProperties);
    }

    private void addProperties(ResourceResolver resourceResolver) {
        @NotNull Resource resource = resourceResolver.resolve(path);
        if(!ResourceUtil.isNonExistingResource(resource)) {
            ResourceFilterStream rfs = resource.adaptTo(ResourceFilterStream.class);
            rfs.setBranchSelector("[jcr:primaryType] == 'cq:Page'")
                    .setChildSelector("[jcr:content/sling:resourceType] == $type")
                    .addParam("type", pageResourceType)
                    .stream()
                    .map(r -> r.getChild(JcrConstants.JCR_CONTENT))
                    .forEach(this::updateProperty);
        }
    }

    private void updateProperty(Resource resultResource) {
        if (reAction == UpdateAction.ADD || reAction == UpdateAction.UPDATE) {
            addOrUpdateProp(resultResource);
        } else if (reAction == UpdateAction.DELETE) {
            removeProp(resultResource);
        }
    }

    private void removeProp(Resource resultResource) {
        try {
            ModifiableValueMap map = resultResource.adaptTo(ModifiableValueMap.class);
            propertyMap.entrySet().stream().forEach(r -> map.remove(r.getKey()));
            resultResource.getResourceResolver().commit();
            recordAction(resultResource.getPath(), reAction.name(), StringUtils.join(propertyMap));
        } catch (PersistenceException e) {
            LOGGER.error("Error occurred while persisting the property {}", e.getMessage());
        }
    }

    private void addOrUpdateProp(Resource resultResource) {
        try {
            ModifiableValueMap map = resultResource.adaptTo(ModifiableValueMap.class);
            propertyMap.entrySet().stream().forEach(r -> map.put(r.getKey(), r.getValue()));
            resultResource.getResourceResolver().commit();
            recordAction(resultResource.getPath(), reAction.name(), StringUtils.join(propertyMap));
        } catch (PersistenceException e) {
            LOGGER.error("Error occurred while persisting the property {}", e.getMessage());
        }
    }

    public enum ReportColumns {
        PATH, ACTION, DESCRIPTION
    }

    private void validateInputs() throws RepositoryException {
        if (sourceFile != null && sourceFile.getSize() > 0) {
            Spreadsheet sheet;
            try {
                sheet = new Spreadsheet(sourceFile, PROPERTY, PROPERTY_TYPE, PROPERTY_VALUE).buildSpreadsheet();
            } catch (IOException ex) {
                throw new RepositoryException("Unable to parse spreadsheet", ex);
            }

            if (!sheet.getHeaderRow().contains(PROPERTY) || !sheet.getHeaderRow().contains(PROPERTY_TYPE) || !sheet.getHeaderRow().contains(PROPERTY_VALUE)) {
                throw new RepositoryException(MessageFormat.format("Spreadsheet should have two columns, respectively named {0}, {1} and {2}", PROPERTY, PROPERTY_TYPE, PROPERTY_VALUE));
            }

            sheet.getDataRowsAsCompositeVariants().forEach(row -> {
                String propertyType = row.get(PROPERTY_TYPE).toString();
                if(StringUtils.equalsAnyIgnoreCase("String", propertyType)) {
                    propertyMap.put(row.get(PROPERTY).toString(), row.get(PROPERTY_VALUE).toString());
                } else if(StringUtils.equalsAnyIgnoreCase("Date", propertyType)) {
                    SimpleDateFormat dt = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS");
                    propertyMap.put(row.get(PROPERTY).toString(), dt.format(row.get(PROPERTY_VALUE).toString()));
                } else if(StringUtils.equalsAnyIgnoreCase("Array", propertyType)) {
                    propertyMap.put(row.get(PROPERTY).toString(), row.get(PROPERTY_VALUE).toString().split(","));
                } else if(StringUtils.equalsAnyIgnoreCase("Long", propertyType)) {
                    Integer result = Optional.ofNullable(row.get(PROPERTY_VALUE).toString())
                            .filter(Objects::nonNull)
                            .map(Integer::parseInt)
                            .orElse(0);
                    propertyMap.put(row.get(PROPERTY).toString(), result);
                } else if(StringUtils.equalsAnyIgnoreCase("Binary", propertyType)) {
                    propertyMap.put(row.get(PROPERTY).toString(), Boolean.valueOf(row.get(PROPERTY_VALUE).toString()));
                }
            });
        }
    }

    List<EnumMap<ReportColumns, String>> reportData = Collections.synchronizedList(new ArrayList<>());

    private void recordAction(String path, String action, String description) {
        EnumMap<ReportColumns, String> row = new EnumMap<>(ReportColumns.class);
        row.put(ReportColumns.PATH, path);
        row.put(ReportColumns.ACTION, action);
        row.put(ReportColumns.DESCRIPTION, description);
        reportData.add(row);
    }

    @Override
    public void storeReport(ProcessInstance instance, ResourceResolver rr) throws RepositoryException, PersistenceException {
        report.setRows(reportData, ReportColumns.class);
        report.persist(rr, instance.getPath() + "/jcr:content/report");
    }
}

Once code is deployed, please go to the following URL and click on start process as shown below:

http://{domain}/apps/acs-commons/content/manage-controlled-processes.html

Start MCP Process

You will see a new process called Property Update as shown below:

Click on the process

Property Updater process

Upload the excel sheet and provide search Path, Action and Resource Type to search for

Configure Process

Excel Sheet:

Upload the excel sheet attached and add the property name, data type and value as shown below:

Property Updater Excel

After executing the process, you can click the completed process and view the results as shown below:

View report
List of updated results with Action

You can also go to CRXDE and check the property getting added to the resources as shown below:

CRXDE results

You can also perform other options by selecting appropriate Action as shown below:

Action Dropdown

You can learn more about MCP here

AEM Content Fragments with GraphQL – Getting started with GraphQL

Problem Statement:

What is GraphQL?

How GraphQL can be used with Content Fragments?

Introduction:

What is GraphQL?

GraphQL is a query language for APIs and provides a complete and understandable description of the data in your API.

For example:

Let’s consider an external system with the following tables:

1 to 1 relationship between Company table with person table and 1 to 1 person table with awards table.

If I need to get all awards

You might be doing a call to

{domain}/api/awards

awards API flow

To get individual person and awards

{domain}/api/persons?personID{ID}&awards={ID}

person API flow

To get individual company and person and awards

{domain}/api/company?companyNam={NAME}personID={ID}&awards={ID}

Company API flow

But in GraphQL you can send the parameters like a query and get all the related content as well

GraphQL API flow

To use Graph QL you need to prepare schemas and based on the schema you can do filter the data.

For more information on GraphQL, you can be visiting the link

Benefits:

  • Avoiding iterative API requests as with REST,
  • Ensuring that delivery is limited to the specific requirements,
  • Allowing for bulk delivery of exactly what is needed for rendering as the response to a single API query.

How GraphQL can be used with Content Fragments?

GraphQL is a strongly typed API, which means that data must be clearly structured and organized by type.

The GraphQL specification provides a series of guidelines on how to create a robust API for interrogating data on a certain instance. To do this, a client needs to fetch the Schema, which contains all the types necessary for a query.

For Content Fragments, the GraphQL schemas (structure and types) are based on Enabled Content Fragment Models and their data types.

Content Fragments can be used as a basis for GraphQL for AEM queries as:

  • They enable you to design, create, curate and publish page-independent content.
  • The Content Fragment Models provide the required structure by means of defined data types.
  • The Fragment Reference, available when defining a model, can be used to define additional layers of structure.
Model References provided by Adobe

Content Fragments

  • Contain structured content.
  • They are based on a Content Fragment Model, which predefines the structure for the resulting fragment.

Content Fragment Models

  • Are used to generate the Schemas, once Enabled.
  • Provide the data types and fields required for GraphQL. They ensure that your application only requests what is possible, and receives what is expected.
  • The data type Fragment References can be used in your model to reference another Content Fragment, and so introduce additional levels of structure.

Fragment References

  • Is of particular interest in conjunction with GraphQL.
  • Is a specific data type that can be used when defining a Content Fragment Model.
  • References another fragment, dependent on a specific Content Fragment Model.
  • Allows you to retrieve structured data.
    • When defined as a multifeed, multiple sub-fragments can be referenced (retrieved) by the prime fragment.

JSON Preview

To help with designing and developing your Content Fragment Models, you can preview JSON output.

Install:

  1. AEM 6.5.11 (aem-service-pkg-6.5.11.zip)
  2. Graph QL OAK Index (cfm-graphql-index-def-1.0.0.zip)
  3. GraphiQL Developer tool (graphiql-0.0.6.zip)

For AEMacS you will get the content fragment with the latest update.

Go to configuration folder

  1. AEM tools section
  2. General selection in sidebar
  3. Configuration bowser

As shown below:

Configuration Folder

Create a configuration folder and select

  1. Content Fragment Models
  2. GraphQL Persistent Queries

As shown below:

Create a Conf folder with required checkboxes

Go to Assets Model:

  1. AEM tools section
  2. Assets selection in sidebar
  3. Content Fragments Model

As shown below:

Go to Assets CF

Select the folder and create the content fragments as shown below:

CF models

You can also install the package attached here

Go to the following URL to access the GraphiQL developer tool and run the following query:

Note: you can also get all the autosuggestions by using the ctrl+space shortcut

{
  cityByPath(_path: "/content/dam/sample-content-fragments/cities/berlin") {
    item {
      _path
      name
      country
      population
      categories
    }
  }
}
GraphiQL developer tool

Download Sample Package here

You can also find more queries and filters in the following link

AEM with Java streams

Problem statement:

How to use java streams in AEM? Can I use streams for iterating and resources?

Requirement:

Use Java streams to iterate child nodes, validating and resources and API’s.

Introduction:

There are a lot of benefits to using streams in Java, such as the ability to write functions at a more abstract level which can reduce code bugs, compact functions into fewer and more readable lines of code, and the ease they offer for parallelization

  • Streams have a strong affinity with functions
  • Streams encourage less mutability
  • Streams encourage looser coupling
  • Streams can succinctly express quite sophisticated behavior
  • Streams provide scope for future efficiency gains

Java Objects:

This class consists of static utility methods for operating on objects. These utilities include null-safe or null-tolerant methods for computing the hash code of an object, returning a string for an object, and comparing two objects.

if (Objects.nonNull(resource)) {
  resource.getValueMap().get("myproperty", StringUtils.EMPTY);
}

Java Optional:

Trying using Java Optional util, which is a box type that holds a reference to another object.

Is immutable and non serializable ant there is no public constructor and can only be present or absent

It is created by the of(), ofNullable(), empty() static method.

In the below example Optional resource is created and you can check whether the resource is present and if present then get the valuemap

Optional < Resource > res = Optional.ofNullable(resource);
if (res.isPresent()) {
    res.get().getValueMap().get("myproperty", StringUtils.EMPTY);
}

you can also call stream to get children’s as shown below:

Optional < Resource > res = Optional.ofNullable(resource);
if (res.isPresent()) {
    List < Resource > jam = res.stream().filter(Objects::nonNull).collect(Collectors.toList());
}

Java Stream Support:

Low-level utility methods for creating and manipulating streams. This class is mostly for library writers presenting stream views of data structures; most static stream methods intended for end users are in the various Stream classes.

In the below example we are trying to get a resource iterator to get all the child resources and map the resources to a page and filter using Objects and finally collect the list of pages.

Iterator < Resource > iterator = childResources.getChildren().iterator();
List < Page > pages = StreamSupport.stream(((Iterable < Resource > )() -> iterator).spliterator(), false)
  .map(currentPage.getPageManager()::getContainingPage).filter(Objects::nonNull)
  .collect(Collectors.toList());

We can also Optional utility to get the children resources or empty list to avoid all kinds of null pointer exceptions.

List < Resource > pagesList = Optional.ofNullable(resource.getChild(Teaser.NN_ACTIONS))
  .map(Resource::getChildren)
  .map(Iterable::spliterator)
  .map(s -> StreamSupport.stream(s, false))
  .orElseGet(Stream::empty)
  .collect(Collectors.toList());

We can also adapt the resource to Page API and call the listchilderens to get all the children and using stream support we are going to map the page paths into a list as shown below:

terator < Page > childIterator = childResources.adaptTo(Page.class).listChildren();
StreamSupport.stream(((Iterable < Page > )() -> childIterator).spliterator(), false)
  .filter(Objects::nonNull)
  .map(childPage -> childPage.getPath())
  .collect(Collectors.toList());

Does this works only on resource and page API?

No, we can also use Content Fragment and other API’s as well for example in the below code we are trying to iterate contentfragment and get all the variations of the contentfragment.

Optional < ContentFragment > contentFragment = Optional.ofNullable(resource.adaptTo(ContentFragment.class));
Iterator < VariationDef > versionIterator = contentFragment.get().listAllVariations();
List < String > variationsList = StreamSupport.stream(((Iterable < VariationDef > )() -> versionIterator).spliterator(), false)
  .filter(Objects::nonNull)
  .map(cfVariation -> cfVariation.getTitle())
  .collect(Collectors.toList());

You can also learn more about other tricks and techniques of Java Streams:

AEM Query builder using Java streams

AEM Get linked Content fragments content

Problem Statement:

We have the following content fragment as part of the AEM

  1. Car details
  2. Agent details

And each car can have multiple agents or agents will be selling multiple cars. How can we link between Cars and Agents CF? and how can we get the linked content onto the page?

Requirement:

Link the Car and Agent CF to maintain the relationship between the content fragments and the same can be pulled into the page and exported.

Introduction:

Content Fragments (CF) allow us to manage page-independent content. They help us prepare content for use in multiple locations/over multiple channels. These are text-based editorial content that may include some structured data elements that are considered pure content without design or layout information. Content Fragments are intended to be used and reused across channels.

Usage

  • Highly structured data-entry/form-based content
  • Long-form editorial content (multi-line elements)
  • Content managed outside the life cycle of the channels delivering it

Create the Car and Agents content fragment models as shown below:

Agent Content fragment
Car Content Fragment

Create a custom component called has linkedcontentfragment as shown below:

Based on the element name condition call the LinkedContentFragment Sling model and also pass the elements to be rendered (based on element names, element data will be pulled).

<template data-sly-template.element="${@ element='The content fragment element'}">
    <div class="cmp-contentfragment__element cmp-contentfragment__element--${element.name}" data-cmp-contentfragment-element-type="${element.dataType}">
        <dd class="cmp-contentfragment__element-value">
            <sly data-sly-test="${element.dataType == 'calendar'}" data-sly-use.tpl="core/wcm/components/contentfragment/v1/contentfragment/calendar.html"
                 data-sly-call="${tpl.element @ date = element.value}"></sly>
            <sly data-sly-test="${element.dataType == 'boolean'}">${element.value ? "true" : "false"}</sly>
            <sly data-sly-test="${element.dataType != 'calendar' && element.dataType != 'boolean' && element.name != 'agentDetails'}">${(element.value) @join='<br/>', context='html'}</sly>
            <sly data-sly-test="${element.name == 'agentDetails'}"
                 data-sly-use.linkedFragment="${'com.mysite.core.models.LinkedContentFragment' @elementValue=element, elements='agentTitle,agentDescription,agentAddress'}">
                data-sly-list.agentData="${linkedFragment.agentsList}">
                <dl data-cmp-data-layer="${agentData.data.json}"
                    data-sly-list.element="${agentData.elements}"
                    data-sly-use.elementTemplate="mysite/components/linkedcontentfragment/element.html">
                    <sly data-sly-call="${elementTemplate.element @ element=element}">
                    </sly>
                </dl>
            </sly>
        </dd>
    </div>
</template>

Create Sling model interface LinkedContentFragment as shown below:

package com.mysite.core.models;

import com.adobe.cq.wcm.core.components.models.Component;
import com.adobe.cq.wcm.core.components.models.contentfragment.ContentFragment;
import java.util.List;

/**
 * Defines the {@code Agent CF Model} Sling Model for the {@code /apps/mysite/components/linkedcontentfragment} component.
 */
public interface LinkedContentFragment extends Component {
    /**
     * Returns the Agents List
     *
     * @return the Content Fragment
     */
    default List<ContentFragment> getAgentsList() {
        throw new UnsupportedOperationException();
    }
}

Create model implementation class called LinkedContentFragmentImpl as shown below, get the element data (String array of paths) and elements to be pulled create a synthetic resource and adapt to core component Content fragment model to pull the element details as well as datalayer (tracking purposes)

package com.mysite.core.models.impl;

import com.adobe.cq.wcm.core.components.models.contentfragment.ContentFragment;
import com.adobe.cq.wcm.core.components.models.contentfragment.DAMContentFragment.DAMContentElement;
import com.adobe.cq.wcm.core.components.util.AbstractComponentImpl;
import com.adobe.granite.ui.components.ds.ValueMapResource;
import com.mysite.core.models.LinkedContentFragment;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ValueMap;
import org.apache.sling.api.wrappers.ValueMapDecorator;
import org.apache.sling.models.annotations.Model;
import org.apache.sling.models.annotations.injectorspecific.InjectionStrategy;
import org.apache.sling.models.annotations.injectorspecific.OSGiService;
import org.apache.sling.models.annotations.injectorspecific.RequestAttribute;
import org.apache.sling.models.annotations.injectorspecific.SlingObject;
import org.apache.sling.models.factory.ModelFactory;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;

@Model(adaptables = SlingHttpServletRequest.class, adapters = {LinkedContentFragment.class}, resourceType = LinkedContentFragmentImpl.RESOURCE_TYPE)
public class LinkedContentFragmentImpl extends AbstractComponentImpl implements LinkedContentFragment {

    public static final String RESOURCE_TYPE = "mysite/components/linkedcontentfragment";
    private static final String CF_DISPLAY_MODE = "displayMode";
    private static final String CF_ELEMENTS = "elementNames";
    private static final String CF_FRAGMENT_PATH = "fragmentPath";

    @RequestAttribute(injectionStrategy = InjectionStrategy.OPTIONAL)
    private DAMContentElement elementValue;

    @RequestAttribute(injectionStrategy = InjectionStrategy.OPTIONAL)
    private String elements;

    @SlingObject
    private ResourceResolver resourceResolver;

    @OSGiService
    private ModelFactory modelFactory;

    @Override
    public List<ContentFragment> getAgentsList() {
        String[] elementList = elements.split(",");
        String[] agentPaths = elementValue.isMultiValue() ? (String[]) elementValue.getValue() : new String[] { (String) elementValue.getValue() };
        List<ContentFragment> elementsFragmentList = new ArrayList<>();
        Arrays.stream(agentPaths).forEach(agentPath -> {
            elementsFragmentList.add(modelFactory.getModelFromWrappedRequest(request, createLinkedSyntheticResource(agentPath, elementList), ContentFragment.class));
        });
        return elementsFragmentList;
    }

    private ValueMapResource createLinkedSyntheticResource(String path, String ... elementList) {
        ValueMap properties = new ValueMapDecorator(new HashMap<>());
        properties.put(CF_ELEMENTS, elementList);
        properties.put(CF_DISPLAY_MODE, "multi");
        properties.put(CF_FRAGMENT_PATH, path);
        return new ValueMapResource(resourceResolver, resource.getPath(), RESOURCE_TYPE, properties);
    }
}

Once we author the car model, we will be pulling the linked agent details as well as shown below:

Component Authoring
Content Fragment rendering

You can also refer to the below link to download the working code

https://github.com/kiransg89/LinkedContentFragment

AEM Content Fragment with Image support

Problem Statement:

How can I get support for Core Image component within content fragment component?

Why should we use Core Image component?

Requirement:

Get support for OOTB image component within content fragment component

Introduction:

The Image Component features adaptive image selection and responsive behaviour with lazy loading for the page visitor as well as easy image placement.

The Image Component comes with robust responsive features ready right out of the box. At the page template level, the design dialog can be used to define the default widths of the image asset. The Image Component will then automatically load the correct width to display depending on the browser window’s size. As the window is resized, the Image Component dynamically loads the correct image size on the fly. There is no need for component developers to worry about defining custom media queries since the Image Component is already optimized to load your content.

In addition, the Image Component supports lazy loading to defer loading of the actual image asset until it is visible in the browser, increasing the responsiveness of your pages.

Create Custom Image content fragment component as shown below and add the following conditions into the element.html file:

Image Contentfragment Component

In the above HTL we are trying to tell if the element name contains the “image” (eg: primaryimage) keyword then don’t print it instead of that call an image model and get synthetic image resource

Note: Make sure your content fragment element name (field name) contains image word

Create a Sling mode ImageContentFragment Interface as shown below:

package com.mysite.core.models;

import org.apache.sling.api.resource.Resource;
import org.osgi.annotation.versioning.ConsumerType;

@ConsumerType
public interface ImageContentFragment {
    /**
     * Getter for Image Synthetic Resource
     *
     * @return resource
     */
    default Resource getImageResource() {
        throw new UnsupportedOperationException();
    }
}

Create a Sling mode implementation ImageContentFragmentImpl class as shown below, this class is used to create a synthetic image resource for Adaptive Image Servlet to work:

package com.mysite.core.models.impl;

import java.util.HashMap;
import javax.annotation.PostConstruct;
import com.mysite.core.models.ImageContentFragment;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ValueMap;
import org.apache.sling.api.wrappers.ValueMapDecorator;
import org.apache.sling.models.annotations.Model;
import org.apache.sling.models.annotations.injectorspecific.InjectionStrategy;
import org.apache.sling.models.annotations.injectorspecific.RequestAttribute;
import org.apache.sling.models.annotations.injectorspecific.SlingObject;
import org.apache.sling.models.annotations.injectorspecific.ValueMapValue;
import com.adobe.cq.wcm.core.components.models.contentfragment.ContentFragment;
import com.adobe.cq.wcm.core.components.models.contentfragment.DAMContentFragment.DAMContentElement;
import com.adobe.granite.ui.components.ds.ValueMapResource;
import com.day.cq.commons.DownloadResource;
import lombok.Getter;

@Model(adaptables = SlingHttpServletRequest.class, adapters = {ImageContentFragment.class}, resourceType = ImageContentFragmentImpl.RESOURCE_TYPE)
public class ImageContentFragmentImpl implements ImageContentFragment{

    public static final String RESOURCE_TYPE = "mysite/components/image";
    private static final String IMAGE = "/image";

    @SlingObject
    private ResourceResolver resourceResolver;

    @SlingObject
    protected Resource resource;

    @ValueMapValue(name = ContentFragment.PN_PATH, injectionStrategy = InjectionStrategy.OPTIONAL)
    private String fragmentPath;

    @RequestAttribute(injectionStrategy = InjectionStrategy.OPTIONAL)
    private DAMContentElement imageElement;

    @ValueMapValue(injectionStrategy = InjectionStrategy.OPTIONAL)
    private String fileReference;

    @Getter
    private Resource imageResource;

    @PostConstruct
    private void initModel() {
        if (StringUtils.isNotEmpty(fragmentPath) && null != imageElement && null != imageElement.getValue()) {
            createSyntheticResource(imageElement.getValue().toString(), resource.getPath() + IMAGE);
        }
    }

    private void createSyntheticResource(String imagePath, String path) {
        ValueMap properties = new ValueMapDecorator(new HashMap<>());
        properties.put(DownloadResource.PN_REFERENCE, imagePath);
        imageResource = new ValueMapResource(resourceResolver, path, ImageContentFragmentImpl.RESOURCE_TYPE, properties);
    }
}

Create an ImageDelegate Lombok based delegation class as shown below, for this example, I am using Image V3 component and this class is used to modify the image srcset method to add image path into the image URL:

package com.mysite.core.models.impl;

import com.adobe.cq.export.json.ExporterConstants;
import com.adobe.cq.wcm.core.components.models.Image;
import lombok.experimental.Delegate;
import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Exporter;
import org.apache.sling.models.annotations.Model;
import org.apache.sling.models.annotations.Via;
import org.apache.sling.models.annotations.injectorspecific.Self;
import org.apache.sling.models.annotations.injectorspecific.SlingObject;
import org.apache.sling.models.annotations.via.ResourceSuperType;
import org.apache.sling.servlets.post.SlingPostConstants;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;

@Model(adaptables = SlingHttpServletRequest.class, adapters = {Image.class}, resourceType = ImageDelegate.RESOURCE_TYPE, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
@Exporter(name = ExporterConstants.SLING_MODEL_EXPORTER_NAME, extensions = ExporterConstants.SLING_MODEL_EXTENSION)
public class ImageDelegate implements Image {
    private static final Logger LOGGER = LoggerFactory.getLogger(ImageDelegate.class);
    public static final String RESOURCE_TYPE = "mysite/components/content/image";
    public static final String CONTENT_DAM_PATH = "/content/dam";
    public static final String HTTPS = "https://";
    public static final String PN_DISPLAY_SIZES = "sizes";
    public static final String WIDTH = "{.width}";

    @SlingObject
    private ResourceResolver resourceResolver;

    @SlingObject
    protected Resource resource;

    @Self
    @Via(type = ResourceSuperType.class)
    @Delegate(excludes = DelegationExclusion.class)
    private Image image;

    @Override
    public String getSrc() {
        return prepareSuffix(image.getSrc());
    }

    @Override
    public String getSrcset() {
        int[] widthsArray = image.getWidths();
        String srcUritemplate = image.getSrcUriTemplate();
        String[] srcsetArray = new String[widthsArray.length];
        if (widthsArray.length > 0 && srcUritemplate != null) {
            String srcUriTemplateDecoded = "";
            try {
                srcUriTemplateDecoded = prepareSuffix(URLDecoder.decode(srcUritemplate, StandardCharsets.UTF_8.name()));
            } catch (UnsupportedEncodingException e) {
                LOGGER.error("Character Decoding failed for {}", resource.getPath());
            }
            if (srcUriTemplateDecoded.contains(WIDTH)) {
                for (int i = 0; i < widthsArray.length; i++) {
                    if (srcUriTemplateDecoded.contains("="+WIDTH)) {
                        srcsetArray[i] = srcUriTemplateDecoded.replace(WIDTH, String.format("%s", widthsArray[i])) + " " + widthsArray[i] + "w";
                    } else {
                        srcsetArray[i] = srcUriTemplateDecoded.replace(WIDTH, String.format(".%s", widthsArray[i])) + " " + widthsArray[i] + "w";
                    }
                }
                return StringUtils.join(srcsetArray, ',');
            }
        }
        return null;
    }

    private String prepareSuffix(String imageSrc) {
        if(StringUtils.isNotEmpty(imageSrc) && !StringUtils.containsIgnoreCase(imageSrc, CONTENT_DAM_PATH)) {
            int endIndex = imageSrc.lastIndexOf(SlingPostConstants.DEFAULT_CREATE_SUFFIX);
            String intermittenResult = imageSrc.substring(0, endIndex);
            endIndex = intermittenResult.lastIndexOf(SlingPostConstants.DEFAULT_CREATE_SUFFIX);
            return intermittenResult.substring(0, endIndex)+image.getFileReference();
        }
        return imageSrc;
    }

    private interface DelegationExclusion {
        String getSrc();
        String getSrcset();
    }
}

In the above code, we are trying to prepend the URL with an image path, because we are using a synthetic image component for the content fragment

For more information on Lombok based component delegation please click on the title to check:

AEM Core Component Delegation

Create a new Adaptive Image servlet and EnhancedRendition class, this servlet is used for displaying appropriate rendition of image component based on width selected based on browser width and pixel ratio:

public class AdaptiveImageServlet  extends SlingSafeMethodsServlet {
    @Override
    protected void doGet(@NotNull SlingHttpServletRequest request, @NotNull SlingHttpServletResponse response) throws IOException {
        try {
            RequestPathInfo requestPathInfo = request.getRequestPathInfo();
            List<String> selectorList = selectorToList(requestPathInfo.getSelectorString());
            String suffix = requestPathInfo.getSuffix();
            String imagePath = suffix;
            String imageName = StringUtils.isNotEmpty(suffix) ? FilenameUtils.getName(suffix) : "";
            Resource component = request.getResource();
            ImageComponent imageComponent = new ImageComponent(component, imagePath);

        } catch (IllegalArgumentException e) {
            LOGGER.error("Invalid image request {}", e.getMessage());
            response.sendError(HttpServletResponse.SC_NOT_FOUND);
        }
    }
    private static class ImageComponent {
        Source source = Source.NONEXISTING;
        Resource imageResource;

        ImageComponent(@NotNull Resource component, String imagePath) {
            if (StringUtils.isNotEmpty(imagePath)) {
                imageResource = component.getResourceResolver().getResource(imagePath);
                source = Source.ASSET;
            }
        }
    }
}

Adaptive Image servlet is a modified version of the Core component Adaptive Image servlet because we are using synthetic image component and Enhanced Rendition class is a support class to get the best image rendition.

Create a simple content fragment Model as shown below:

Make sure the contentreference field name contains the image

Sample Content Fragment Model

Create a new content fragment under asset path (/content/dam/{project path}) as shown below:

Content Fragment

Create a sample page and add the content fragment component and select all the fields as a multifield as shown below:

Custom Content Fragment Component Authoing

As we can see we are able to fetch Core component Image component with Image widths configured into Image component policy (design dialog)

Content Fragment with OOTB Image

You can also get the actual working code from the below link:

https://github.com/kiransg89/ImageContentFragment