Many SEO and Web diagnostic tools like Screaming Frog and Deep Crawl allow you to export XML site maps and lists of URL’s containing all of the clean URL’s. You can use these files to augment your existing CMS generated site maps to ensure you have maximum coverage.
Follow the instruction for “Setup Auto Update” if you don’t not have a current Auto Update project in place.
Copy the URL for the site map location in Dropbox by right clicking on the file name and clicking the option for “Copy Dropbox Link” and it will be copied to your clipboard. Dropbox instructions
Step 1: Go to the XML site map option in Auto Update and paste your link into the box. Repeat these steps until all of the unique links are added to HREFLang Builder.
Step 2: In this step you need to convert the links from view to download. At the end of each of the files it should end with ?dl=0. For example if this is my the file I added – https://www.dropbox.com/s/fpzagugegpwme5h/us-en-sitemap.xml?dl=0 I need to covert this to ?dl=1. by replacing the 0 with a 1. If you haven’t already, install the Dropbox desktop app on your computer.
Step 3: Save and update by clicking the blue “Create” button.
Once the project is updated, you can wait until the next normal schedule run or you can run it immediately by clicking the run button. We suggest doing this immediately as any URL’s that did not copy correctly will break and not be updated.
As long as your set up Screaming Frog crawls and scheduler correctly it will update this file automatically allowing you to have fresh data.