Summarizing Subscriber Activity

Here’s a scalable method I’ve used to aggregate subscriber-level engagement data. I’ve used the results of this data to populate high/low engagement audiences. I will typically drop this in as soon as I can — just to get the data accumulating. It’s intended to run daily.

Automation

Subscriber_Activity_Summary

Step 1 – SQL Query: Subscriber_Activity_Summary_Status

Step 2 – SQL Query: Subscriber_Activity_Summary_Sends

Step 3 – SQL Query: Subscriber_Activity_Summary_Opens

Step 4 – SQL Query: Subscriber_Activity_Summary_Clicks

Step 5 – SQL Query: Subscriber_Activity_Summary_MostRecentInteraction

Data Extensions

Subscriber_Activity_Summary

Name Data Type Len PK Req’d Default
SubscriberId Number 0 true true
EmailAddress EmailAddress 254 false false
SubscriberKey Text 254 false false
Status Text 15 false false
SendCount Number false false 0
LastSend Date false false
OpenCount Number false false 0
LastOpen Date false false
ClickCount Number false false 0
LastClick Date false false
mostRecentInteraction Date false false
InsertedDate Date false false GetDate()

Queries

Subscriber_Activity_Summary_Status

NOTE: Add the ent. prefix on _Subscribers if you’re in a child BU.

select
  s.subscriberid
, s.subscriberkey
, s.emailAddress
, s.status
from _Subscribers s
/* name: Subscriber_Activity_Summary_Status */
/* target: Subscriber_Activity_Summary */
/* action: update */

Subscriber_Activity_Summary_Sends

select
  x.subscriberid
, x.subscriberkey
, isnull(sas.SendCount,0) + x.SendCount SendCount
, x.LastSend
from (

    select
      y.subscriberid
    , y.subscriberkey
    , count(*) SendCount
    , max(y.eventDate) LastSend
    from (
        select
          a.subscriberid
        , a.subscriberkey
        , a.eventDate
        from _Sent a
        where a.EventDate >= convert(date, getDate()-1)
        and a.EventDate < convert(date, getDate())
    ) y
    group by
    y.subscriberid
    , y.subscriberkey

) x
left join Subscriber_Activity_Summary sas on sas.subscriberid = x.subscriberid
where
sas.subscriberid is null
or x.lastSend > isnull(sas.lastSend,'1980-01-01')
/* name: Subscriber_Activity_Summary_Sends */
/* target: Subscriber_Activity_Summary */
/* action: update */

Subscriber_Activity_Summary_Opens

select
x.subscriberid
, x.subscriberkey
, isnull(sas.OpenCount,0) + x.OpenCount OpenCount
, x.LastOpen
from (

    select
      y.subscriberid
    , y.subscriberkey
    , count(*) OpenCount
    , max(y.eventDate) LastOpen
    from (
        select
          a.subscriberid
        , a.subscriberkey
        , a.eventDate
        from _Open a
        where a.EventDate >= convert(date, getDate()-1)
        and a.EventDate < convert(date, getDate())
    ) y
    group by
    y.subscriberid
    , y.subscriberkey

) x
left join Subscriber_Activity_Summary sas on sas.subscriberid = x.subscriberid
where
sas.subscriberid is null
or x.lastOpen > isnull(sas.lastOpen,'1980-01-01')
/* name: Subscriber_Activity_Summary_Opens */
/* target: Subscriber_Activity_Summary */
/* action: update */

Subscriber_Activity_Summary_Clicks

select
x.subscriberid
, x.subscriberkey
, isnull(sas.ClickCount,0) + x.ClickCount ClickCount
, x.LastClick
from (

    select
      y.subscriberid
    , y.subscriberkey
    , count(*) ClickCount
    , max(y.eventDate) LastClick
    from (
        select
          a.subscriberid
        , a.subscriberkey
        , a.eventDate
        from _Click a
        where a.EventDate >= convert(date, getDate()-1)
        and a.EventDate < convert(date, getDate())
    ) y
    group by
    y.subscriberid
    , y.subscriberkey

) x
left join Subscriber_Activity_Summary sas on sas.subscriberid = x.subscriberid
where
sas.subscriberid is null
or x.lastClick > isnull(sas.lastClick,'1980-01-01')
/* name: Subscriber_Activity_Summary_Clicks */
/* target: Subscriber_Activity_Summary */
/* action: update */

Subscriber_Activity_Summary_MostRecentInteraction

select
  a.subscriberid
, a.subscriberkey
, case
      when a.lastOpen > coalesce(a.lastClick, convert(datetime, '1980-01-01')) then a.lastOpen
      else a.lastClick
  end mostRecentInteraction
from Subscriber_Activity_Summary a
where a.lastOpen is not null
or a.lastClick is not null
/* name: Subscriber_Activity_Summary_MostRecentInteraction */
/* target: Subscriber_Activity_Summary */
/* action: update */

Storing File Import Results in a Data Extension

I don’t know about you, but I’ve found monitoring crucial File Import Activities in SFMC a challenge. I think it’s important to ensure all of the rows in the file got imported or have some recourse to identify exceptions — outside of accessing the results file that gets dropped in the SFTP Import folder or relying on the import results email.

Here’s a script that retrieves and iterates through the ImportResultsSummary SOAP object data (via WSProxy) and writes it to a Data Extension of the same name.

NOTE: The DestinationIDis either a ListID or the ObjectID of a Data Extension. If you’ve got an inventory of DataExtension SOAP object data, you can use something like this dataset to build a pretty comprehensive report on import exceptions.

I started setting proper Customer/External Keys on all Import Activities after using this data, just so ImportDefinitionCustomerKey is meaningful. One less thing to have to track down when troubleshooting.

Of course, I welcome any feedback y’all might have on this.

Schema

Code

Storing SFMC AuditEvents in a Data Extension

Audit Events are worthwhile to explore in Salesforce Marketing Cloud (SFMC) if you’re in a vertical that includes tighter roles and permissions. You can supplement your audit scheme by retrieving these Audit Events via REST and writing them to a Data Extension (DE).

First, you’ll need to enable the feature here: Email Studio > Admin > Security Settings > Enable Audit Trail Data Collection > Yes

After you’ve collected data for a few days, create a Script Activity like this and set it to run on the desired interval in an Automation.

You’ll need to add your target Data Extension’s Category ID, MID, REST API Tenant, Client Id and Client Secret.

SSH Keys for SFTP in SFMC

If you want to use your own SSH keys for use in SFTP transfers to and from SFMC, then here’s what you need to know.

SFMC pushing files to third-party SFTP via SSH

SFMC Third-party

Private SSH Key

Specifications

  • RSA
  • 2048-bit
  • OpenSSH format
  • Can be created with or without passphrase using with ssh-keygen or puttygen

Installation

  • SSH option must be enabled by SF Support
  • Upload key file as SSH type in Setup > Admin > Data Management > Key Management

Public SSH Key

Specifications

  • See Private SSH Key specifications

Installation

  • Specific to third-party key management system

Third-party pushing files to SFMC SFTP via SSH

SFMC Third-party

Public SSH Key

Specifications

  • See Private SSH Key specifications

Installation

  • Create a new SFTP Account for connection in Setup > Admin > Data Management > FTP Accounts
  • SSH option must be enabled by SF Support
  • Upload key file as SSH type in Setup > Admin > Data Management > Key Management

Private SSH Key

Specifications

  • RSA
  • 4096-bit
  • PEM format only
  • With or without passphrase
  • Can be created with ssh-keygen or puttygen

Installation

  • Specific to third-party key management system

 

For the first scenario (pushing files from SFMC to a third-party server), here’s how you can generate an SFMC-friendly SSH key pair with ssh-keygen.
Just simply open a command prompt and issue these commands (this is specific to Windows, but it’s very similar on a Mac):

> ssh-keygen -t rsa -b 2048
Generating public/private rsa key pair.
Enter file in which to save the key (C:\DEFAULTUSERPATH/.ssh/id_rsa): c:\YOURDESTINATIONPATH\testkey
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in c:\YOURDESTINATIONPATH\testkey.
Your public key has been saved in c:\YOURDESTINATIONPATH\testkey.pub.
The key fingerprint is:
(redacted)
> dir /b testkey*
testkey
testkey.pub
> ren testkey testkey.ppk

 

For the second scenario, (pushing files from a third-party server to SFMC), here’s how to get it in the correct format for SFMC with ssh-keygen:

> ssh-keygen -t rsa -b 4096 -m PEM
Generating public/private rsa key pair.
Enter file in which to save the key (C:\DEFAULTUSERPATH/.ssh/id_rsa): c:\YOURDESTINATIONPATH\testkey
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in c:\YOURDESTINATIONPATH\testkey.
Your public key has been saved in c:\YOURDESTINATIONPATH\testkey.pub.
The key fingerprint is:
(redacted)
> dir /b testkey*
testkey
testkey.pub
> ssh-keygen -e -f testkey.pub > testkey-ssh2.pub
> ren testkey testkey.ppk

Automation Status Across Business Units

If you need to get a status snapshot of Automations across business units, here’s one way to go about it.

Add this code to a CloudPage in the parent business unit, update the mids array to include the desired values and publish.

You could also set something like this up in a Script Activity and write the results to a Data Extension for use in another process.

NOTE: As leto96 mentioned in the comments, CloudPages are public and should not be considered secure. Anyone with the URL can access it. Initial SSJS development is best done in a CloudPage, where you have a way to debug with output. Just be careful to unpublish or secure any utility/reporting type pages after you’re done.