Migrating Drupal 7 Image Files to Drupal 8 Media Images (no PHP required!)

April 24, 2019 Doug Chan

I’m new to migrate. And it’s a beast. But I think after a few dozen hours of diligence, I think I’ve figured out my preferred method.

This post is for someone who already understands how to use views, develop custom modules, and knows how to use drush migration operations.

I originally tried using migrate_drupal, which honestly did a great job at migrating my user roles, users, and taxonomies. But the site I am migrating is 5 years old with a good amount of technical debt having been touched by over 10 (and maybe even 20) developers over its lifetime, so I think it’s time for a fresh start. I want to migrate everything else (content types, files, etc) manually.

All the Drupal 7 content types were using image (file) fields, but I want the new Drupal 8 site to use media fields. I couldn’t use the migration provided by migrate_drupal, so I had to get creative.

This will be a two step process:

Use D7 views_datasource module to create JSON endpoints on my old site (and since we’re using a JSON source, you can honestly migrate from anything).
Write a custom migration in D8.

Drupal 7 and Views JSON

Create a views JSON that pulls all file content with the following fields:

File ID (this will become the Media ID — we want to keep the IDs so when we merge content, it’ll be easier to also migrate their references)
Name
Absolute path (e.g. https://fullpath.com/sites/default/files/something.jpg)
Alt text
Date
User ID (Many users in CMS — I want to make sure they keep ownership of their images)
Title text (optional and not really used, but some people have taken the time to enter this field so I figure it would be worth migrating over)

Also, in the format settings, make sure you set a root object name (I used “images”) and leave the the “Top-level child object” field empty.

Now, your JSON object should look something like this:

{
  "images": [
    {
      "fid": "6",
      "filename": "your-image.jpg",
      "path": "https://youroldsite.com/sites/default/files/your-image.jpg",
      "alt": "People shaking hands",
      "created": "1432021142",
      "uid": "1",
      "title": ""
    },
    {
      "fid": "11",
      "filename": "your-screenshot.jpg",
      "path": "https://youroldsite.com/sites/default/files/your-screenshot.jpg",
      "alt": "Video demo",
      "created": "1432021142",
      "uid": "1",
      "title": ""
    }
  ],
  "pager": {
    "pages": null,
    "page": null,
    "count": 0,
    "limit": 0
  }
}

{

"images": [

{

"fid": "6",

"filename": "your-image.jpg",

"path": "https://youroldsite.com/sites/default/files/your-image.jpg",

"alt": "People shaking hands",

"created": "1432021142",

"uid": "1",

"title": ""

{

"fid": "11",

"filename": "your-screenshot.jpg",

"path": "https://youroldsite.com/sites/default/files/your-screenshot.jpg",

"alt": "Video demo",

"created": "1432021142",

"uid": "1",

"title": ""

}

"pager": {

"pages": null,

"page": null,

"count": 0,

"limit": 0

}

The Drupal 8 Migration

Now for the beast. On your new site, install the following contrib modules:

media (core)
migrate (core)
migrate_plus (contrib)
migrate_tools (contrib)

You’ll also need to apply this patch that fixes a problem with the entity_generate plugin in order to get UIDs and FIDs to import correctly into your generated file entities https://www.drupal.org/project/migrate_plus/issues/2975266

You’ll then need two files in a custom module:

my_module/config/install/migrate_plus.migration_group.files.yml
my_module/config/install/migrate_plus.migration.images.yml

# migrate_plus.migration_group.files.yml
langcode: en
status: true
module: my_module
id: files
label: File imports
description: Import files from my old website.
source_type: JSON
dependencies:
  enforced:
    module:
      - my_module

# migrate_plus.migration_group.files.yml

langcode: en

status: true

module: my_module

id: files

label: File imports

description: Import files from my old website.

source_type: JSON

dependencies:

enforced:

module:

- my_module

# migrate_plus.migration.images.yml
id: images
label: Files
migration_group: files
migration_tags:
  - files
source:
  plugin: url
  data_fetcher_plugin: http
  data_parser_plugin: json
  urls:
    - 'https://youroldsite.com/api/to/images'
  item_selector: images
  fields:
    -
      name: fid
      label: 'File ID'
      selector: fid
    -
      name: filename
      label: 'Name'
      selector: name
    -
      name: path
      label: 'Path'
      selector: path
    -
      name: alt
      label: 'Alt'
      selector: alt
    -
      name: created
      label: 'Created'
      selector: created
    -
      name: uid
      label: 'User ID'
      selector: uid
    -
      name: title
      label: 'Title text'
      selector: title
  ids:
    fid:
      type: integer
process:
  # Preparation.
  destination_basename:
    plugin: callback
    callable: basename
    source: path
  destination_basepath:
    plugin: default_value
    default_value: public://
  destination_path:
    plugin: concat
    source:
      - '@destination_basepath'
      - '@destination_basename'

  # Mapping.
  mid: fid
  bundle:
    plugin: default_value
    default_value: image
  name: name
  'field_media_image/title': title
  'field_media_image/alt': alt
  'field_media_image/target_id':
    -
      plugin: skip_on_empty
      method: process
      source: path
    -
      plugin: file_copy
      source:
        - path
        - '@destination_path'
    -
      plugin: entity_generate
      values:
        uid: uid
        fid: fid
  status:
    plugin: default_value
    default_value: 1
  created: created
  changed: created

destination:
  plugin: entity:media

# migrate_plus.migration.images.yml

id: images

label: Files

migration_group: files

migration_tags:

- files

source:

plugin: url

data_fetcher_plugin: http

data_parser_plugin: json

urls:

- 'https://youroldsite.com/api/to/images'

item_selector: images

fields:

label: 'File ID'

selector: fid

label: 'Name'

selector: name

label: 'Path'

selector: path

label: 'Alt'

selector: alt

label: 'Created'

selector: created

label: 'User ID'

selector: uid

label: 'Title text'

selector: title

ids:

fid:

type: integer

process:

# Preparation.

destination_basename:

plugin: callback

callable: basename

source: path

destination_basepath:

plugin: default_value

default_value: public://

destination_path:

plugin: concat

source:

- '@destination_basepath'

- '@destination_basename'

# Mapping.

mid: fid

bundle:

plugin: default_value

default_value: image

'field_media_image/title': title

'field_media_image/alt': alt

'field_media_image/target_id':

plugin: skip_on_empty

method: process

source: path

plugin: file_copy

source:

- path

- '@destination_path'

plugin: entity_generate

values:

uid: uid

fid: fid

status:

plugin: default_value

default_value: 1

created: created

changed: created

destination:

plugin: entity:media

Next Steps

Install the your custom module, and you should be good to go with a drush mim images.

I’m going to use this to also migrate PDFs and other documents. I thought about creating one huge migration to migrate all file entities to all media entities, but I figure it would be cleaner (and more maintainable) to do one bundle at a time.

This took me longer than I’d like to admit to get together, but hopefully this works for you like it did for me. Feel free to drop a comment for any comments/questions. Good luck!