Import media files

Media can be imported to the VIP File System with VIP-CLI. This is especially useful for migrating from an existing WordPress site to a site hosted on the VIP Platform.

Limitations

• Media can only be imported to a production environment. Importing media to non-production environments is not necessary, as media uploaded to production is immediately made available to the child environments as well.
• Only one import per environment can be run at a single time. Attempts to start a second import while a previous import is still in progress will result in an error.
• The size of the entire archive that holds the files to be imported cannot be more than 30 GB.
• The maximum size for individual media files imported to VIP’s File System is 1GB.
• After a file has been imported, it cannot be deleted from the File System.

Directory structure

The directory to be imported must be named uploads.

In most cases, media uploaded to a WordPress install is stored in a directory uploads/, which is a subdirectory of wp-content/. To prepare media files for import to VIP, an archived copy of the uploads/ directory is recommended.

If a non-VIP WordPress site has customized the name of the uploads/ directory, an extra step will be needed to duplicate the directory to a working location and re-name the duplicate directory to uploads before importing the media to VIP.

When an archive’s directory structure contains a directory named uploads/, only the files within that directory will be imported and all other files will be ignored.

If no directories named uploads/ exist within an archive, the import tool will import all contents of the archive with their existing directory structure.

Multisites

Media import directories for multisites must also be named uploads.

However the sub-directory structure of the uploads/ directory for sites belonging to a WordPress multisite will differ from WordPress single sites. The uploads/ file path for the root site (Site ID 1) of the multisite will be uploads/, but all subsequent sites on your network will be in the uploads/sites/ directory structure, in their own subfolder based on the site number, designated by the database.

Upload paths for non-root sites follow the pattern uploads/sites/{SITE ID}/{YYYY}/{MM}/{FILE NAME}, and values for {SITE ID} do not have any leading 0’s (e.g., use 2, not 02).

Special characters in filenames

WordPress Core sanitizes file names, removing or replacing special characters upon upload to the media library, for example:
 ~?,[]/\=<>:;'\&$#*()|~`!{}%+ and whitespace ().

Media file imports to VIP bypass the WordPress sanitize_file_name() function, and all special characters are allowed, for example:
spaces ( ) [ ] ~ + & # % = ' ” \ < > : ; , / ? $ * | ` ! { } .
Encoded or alternate whitespace, such as %20or + will be converted to proper spaces during the import.

Intermediate images

Traditional installations of WordPress create intermediate images, an array of smaller versions of each uploaded image. The generated intermediate files are unnecessary on VIP, as the VIP File System is able to generate and cache any requested dimension of the originally uploaded image. This reduces the amount of storage a site requires and provides image size flexibility at no loss of speed. It is recommended that intermediate images are excluded from the imported media directory.

Intermediate images can be imported using the -i flag on the command. Including intermediate images in an import will cause the process to take a longer time to complete.

Unsupported file types

Imports to the VIP File System will fail for the following file types:

• Files without extensions
• Files not included in the list of file types supported by the VIP File System
• Hidden files or or files within hidden directories will be ignored. Hidden files will not be included in the count of files that are discovered by the import media command, and they will not be included in the final count of files that are successfully imported.

SVGs

SVG files allow embedded JavaScript, which can pose a security risk. If SVGs exist in the uploads/ directory that will be imported, it is recommended to scan the file contents prior to media import. The file contents of an SVG can be scanned by:

• examining the contents within a file editor (Visual Studio Code, for example)
• a tool such as http://svg.enshrined.co.uk/ or https://github.com/Automattic/vip-go-svg-sanitizer
• a PHPCS scan using the WordPress-VIP-Go standard

All SVG files introduced to a GitHub repository in a pull request will undergo an SVG analysis as part of the automated PHPCS scan using the WordPress-VIP-Go standard from the VIP Code Analysis Bot. Any discovered irregularities will be reported in the automated code review.

Import media files with VIP-CLI

Prior to running any import commands, confirm all items in this checklist:

1. Import files are archived (.tar.gz, .tgz, .tar.bz2, or .zip).
2. The archive decompresses to a directory named uploads , and only contains the media required for your site(s).
3. Import files are uploaded to a publicly accessible URL (e.g., https://www.example.com/uploads.tar.gz).
   • File-sharing services such as Dropbox, Box, or WeTransfer are able to provide publicly accessible URLs.
   • A downloadable Dropbox URL includes a ?dl=1 query parameter.
   • Google Drive URLs are not supported.

VIP-CLI commands for media import

Initiate the import of the contents of the archive to a production environment:

vip import media @mysite.production "https://www.example.com/uploads.tar.gz"

Add the --overwriteExistingFiles option (or -o ) to force the silent overwrite of existing files during import:

vip import media @mysite.production "https://www.example.com/uploads.tar.gz" --overwriteExistingFiles

During the import process, the CLI will display the real-time progress of the import. If desired, the real-time progress of an import can be hidden using the Ctrl + C key combination.

When all files in the archive have been processed, it will return with a status report saved in the current working directory. The exact path to this status report will be shown on screen.

Check on the status of an import

A text status report on progress can be requested during the import:

vip import media status @mysite.production

Formatting logs for failed file imports

If one or more files fail to import, an error log will be created on the local machine at the end of an import. Error logs include a list of failed files and the reasons for their failure. Error logs are unable to record more than 400,000 file failures.

By default, import logs are created in .txt format. Add the --exportFileErrorsToJson option (or -E) to write the error logs to a .json file instead.

vip import media @mysite.production "https://www.example.com/uploads.tar.gz" -E

Abort an ongoing Import

The import process can be aborted after the process has begun. Any files imported up until the abort command was executed cannot be deleted.

vip import media abort @mysite.production

Additional commands and options

Refer to the help menu for additional options, commands, and examples:

vip import media --help