Drupal 8 Salesforce Suite Roadmap: On Track for Beta Release

This post is the second in a series about Message Agency's commitment to ensuring that nonprofits can continue to leverage Salesforce and Drupal with every new release. We developed the original integration modules in 2010 for Drupal 6 and have continued to support this important tool ever since. We've taken the lead in porting the modules for Drupal 8 and have committed to accelerate development, with a release candidate planned for the first quarter of 2017. This post, by Aaron Bauman, reports our progress.

Since my last post in October, we've made some real headway on our Salesforce Suite 8.x release. At this point we're past the "porting" phase and blazing new territory. Nearly all the features from 7.x have been implemented in Drupal 8, and we're working on improving them and adding some cool new features.

On Schedule!

Focusing 50% of my time on module development allowed me to put some real effort into this module. During the past few months, I've helped onboard 2 teammates, Alex Rhodes and David Kosbob, who've quickly become productive. Internally, we're giving the project first class resources, including a project manager, sprint planning, and proper allocations. We've set ourselves up for success, and it's paid off in terms of momentum and efficiency.

We intend to release a beta by the end of January and a release candidate by early March.  

Moving to Git.Drupal.org

For code management, we're running the project through using a branching/pull-request strategy. All development happens on a branch, and all commits are peer-reviewed via pull requests. Adding git.drupal.org as a remote upstream has made it easy to fold in contrib-space patches as well; shout out to evanjenkins, bezhermoso, and gcb for their contributions.

New Features

We've achieved some major milestones so far, including:

• Mapping UI overhaul.  Salesfore Mapping Fields now enjoy their own plugin system, allowing for maximum extensibility. For example, "Record Type" is now its own mapping field plugin type, rather than receiving special treatment in the push and pull systems.
   
• Objectification of Salesforce resources.  Moving in the direction of a proper REST PHP SDK, we now have proper classes for Query Result, SObject, Salesforce ID, various REST Responses, and others. This not only allows for simple type-hinting across other classes, but also gives developers consistent and reliable paves the way for even greater extensibility in the future.
   
• Push queue overhaul, and cron-based push.  Drupal 7's asynchronous push left a lot to be desired. Lack of error handling made debugging and troubleshooting difficult to impossible. Lack of optimizations burned unnecessary API calls. Both of these limitations were imposed by Drupal Queue API's fundamental nature. In Drupal 7, our options for extending the Queue system were limited. In Drupal 8, we've implemented a Salesforce Push Queue service, building on Drupal's DatabaseQueue. We've taken the opportunity to normalize queue items, optimize queue operations, and implement error handling and recovery.
   
• Cron-based pull. The cron-based pull queue is also overhauled, but performs completely based on Drupal's core Queue API for scheduled synchronizations from Salesforce to Drupal.
   
• Real-time push, and entity-based push/pull form.  In addition to asynchronous operations, we've maintained real-time REST-based push for entity CRUD operations, as well as a standalone form for one-off pushes and pulls.

On the horizon, we've got plenty of worked lined up, including:

• SOAP client for batch push operations.  Drupal administrators shouldn't need to worry about the difference between SOAP and REST. Our goal is to use SOAP when it makes sense, and REST for everything else. The 8.x implementation of a SOAP API client will be part of Salesforce's core module, and the choice between SOAP and REST opaque the administrators.
   
• Tests. Testing 3rd-party web services can be tricky, and requires careful planning and mocking. A stable Salesforce 8.x release will include test coverage for push and pull operations using mock REST and SOAP endpoints, allowing for proper regression testing and test-driven development.
   
• Migration. Our stable release will also include some means for Drupal 7 users to move their solutions into Drupal 8.

Our funder has also proposed some new feature ideas, which will make their way into our development pipeline in advance of a stable release, including:

• Queue on failure. Attempt push synchronization immediately on entity save; enqueue for asynchronous push only on failure. This feature idea is a great compromise between the current binary sync/async decision point.
   
• Queue settings per mapping. Allow administrators to assign sync intervals per-mapping, instead of running all sync operations on every cron run. This feature idea will allow administrators to tweak their synchronizations according to business needs, without the need to implement extensive hook-based logic.
   
• Batch op UI. Give administrators a UI to trigger batch push and pull operations. Leveraging Drupal core views and bulk operations will give administrators a powerful way to synchronize data, especially to seed data for deployments, or to re-sync during error recovery.
   
• Queue UI. Provide visibility into scheduled async operations by giving administrators a UI for push and pull queues. Again, since we're using Queue API standards, we'll be able to realize a powerful interface with minimal effort.

Heavy Lifting

By the numbers, so far:

• 5 contributors including 2 Message Agency staff
• Merged 7 major branches
• More than 100 commits
• Nearly 300 hours logged across 5 Message Agency dev and PM staff, and 3 drupal.org users