Delete all the assets which don’t have references to improve AEM performance in turn Indexes and search/query performance.
Introduction:
How do assets get published?
The author uploads the images and publishes the assets
Create a launcher and workflow which processes assets metadata and publishes the pages
Whenever we publish any pages and if the page has references to assets, then during publishing, it asks to replicate the references as well.
What happens when the page is unpublished?
When the page is deactivated, assets referenced to the page will not be deactivated because this asset might have reference to the other pages hence out of the box assets won’t be deactivated.
If we perform cleanup, deactivate and delete old pages, we might not be cleaning up assets related to this page.
What advantages of cleaning up of old assets?
Drastically reduces repository size
Improves DAM Asset search
Improves indexing
Generate Published Asset Report by visiting:
Go to Tools -> Assets -> Reports as shown below:
Asset Report Tool Section
Click on create and click on Publish report
Asset Publish Report
Provide folder path and start date and end date
Asset Report Configure Page
Select the columns as per the requirement
Asset Report Custom Column Page
Finally, the report will be ready with all the assets lists as shown below
Asset Report
Download the report to see the final list of images
DAM Report result
To generate the report let’s create a new AEM tool:
AEM OOTB comes with multiple tools in AEM and to access all the tools you need to navigate to the tool section and select the appropriate sections to perform all the operations
For example:
AEM operations
Managing templates
Cloud configurations
ACS Commons tools etc..
Tools are an essential part of AEM and avoid any dependency on Groovy scripts or any add scripts and can be managed or extended at any given point in time.
Broken Asset Report generates a report of all the unreferenced assets by running a Reference Search query across the repository every 30s (by default), you can update the scheduler expression based on your repository size.
The scheduler also checks for the CPU/HEAP size before triggering the reference search process and for more details on Throttled scheduler please refer to the link.
Once the repository is built and deployed, you will be able to access the Broken Asset Reference report as shown below:
Broken Asset Reference Tool Section
You can select the report just now created by you in the drop down as shown below:
Select the Asset Report from dropdown
Provide scheduler expression as per your needs and select the result refresh interval for every 10s or as per your needs
You can see the results as shown below once the process is kicked off and it will also show the current row its processing and also CPU/Heap usage.
Reference Search Running status
For some reason, if your system CPU/Heap is throttling then from the backend it takes care of not running your scheduler or you can also manually unschedule the scheduler.
Once your system’s CPU comes back to normal you can go back and select the report and schedule again and report generation picks from the current row where it was left off.
Once the processing is complete click on the report name to download the generated report.
Reference Search Completed StatusReport Excel with Has reference Column
Generate the Splunk (logs) results by running a query to get all the assets to call (/content/dam) requests on dispatcher/publisher from the past 1year or so.
You can also reach out to the Analytics team, requesting image impressions (data on image usage) from the past 1 year or so.
Please provide your valuable feedback in the comments.
Get a List of all the Assets which are missing references
Requirement:
Get the list of broken asset references to unpublish and remove them repo to improve the system stability and performance.
Introduction:
How do assets get published?
The author uploads the images and publishes the assets
Create a launcher and workflow which process assets metadata and publish the pages
Whenever we publish any pages and if the page has references to assets, then during publishing, it asks to replicate the references as well.
What happens when the page is unpublished?
When the page is deactivated, assets referenced to the page will not be deactivated because this asset might have reference to the other pages hence out of the box assets won’t be deactivated.
If we perform cleanup, deactivate and delete old pages, we might not be cleaning up assets related to this page.
Advantages of cleaning up old assets?
Drastically reduces repository size
Improves DAM Asset search
Improves indexing
Get Publish Report using Assets Report:
Go to Tools -> Assets -> Reports as shown below:
Asset Reports
Click on create and click on Publish report
Select Publish report
Provide folder path and start date and end date
Add Report details
Select the columns as per requirement
Configure columns for the report
Finally, report will be ready with all the assets lists as shown below
Completed Reports
Download the report to see the final list of images
Example Report CSV file
If Images are unpublished then we can ask authors to review and delete them
If images are published but has no references to figure this out, we need a new process.
MCP (Manage Controlled Processes) is both a dashboard for performing complex tasks and a rich API for defining these tasks as process definitions. In addition to kicking off new processes, users can also monitor running tasks, retrieve information about completed tasks, halt work, and so on.
Add the following maven dependency to your pom to extend MCP
After building the code you can see the new Process showing up in MCP
Borken Asset Refernce Process
Copy the Path column into the new Excel sheet as shown below
Path column into new excel file
Upload into the process and start to see all the images which are published yet unreferenced as shown below
Why does Chunk count?
Chunk count helps the SQL 2 query to group by the paths, which will be maxing 4500 and it won’t take more than that (configurable based on the environment). However basically, if we have 20000 / 4500 = 4.44 ~ 5 we will be running the query max five times to generate the below report
Share the report with the content authors team to validate if images are required if not plan to clean up using
After completing the processExample report and we can download and share with Authors
Clean up Process:
Authors don’t have to unpublish and delete individual images, then can you use the below process to upload the excel sheet with all the approved image paths and upload it to the process to deactivate and delete