Andrew: Okay, let's begin.
So for those just joining, welcome to the FME in Education:
Automating Data Flows with FME Server and FME Cloud webinar.
This webinar is presented by Plaid Consulting,
a firm that helps higher education institutions
use data more effectively.
Safe Software is the vendor of FME and a partner of ours.
This is the third in a series
of webinars focused on using FME in education.
Previous topics included accelerating
and improving your data analysis and combining data
from web based and on campus systems.
Those are available on our website as shown below.
Today's topic focuses on automating data flows
with FME Server and FME Cloud.
At this point in the pandemic,
I'm sure you're all very familiar with Zoom.
There is a few things to be aware of.
We're recording this webinars
so that it'll be available on our website afterwards.
You'll get an email about that automatically
with a link to it when it's ready in a day or two.
If you have questions, feel free to use the chat
or the Q&A, whichever is most preferential for you.
Occasionally, but not today, there are polls in Zoom.
Those will pop up on the screen
if and when they're relevant.
Plaid Consulting provides data integration research
and analysis services for higher education institutions
to improve their policies, processes, and services.
So today we are aiming to talk about
what is FME Server and what is FME Cloud.
We're gonna show some demonstrations
of both automations and server applications.
I'm Andrew Drinkwater.
I'm a co founder of Plaid Consulting.
Plaid is about four years old.
And as mentioned, we focus on
helping educational institutions use data more effectively.
Prior to starting Plaid, I worked
as a senior planning research analyst
at the University of British Columbia in Vancouver, Canada.
My business partner, Patrick Lougheed, fellow co-founder,
is also presenting today.
You'll hear from him shortly.
Prior to founding Plaid, Pat worked
as the Director of Business and Policy Analysis
at Simon Fraser University in Vancouver, Canada,
responsible for many aspects of the data realm
related to student services and beyond.
A lot of what we're gonna talk
about today is data integration
and data integration involves combining data
that reside in different places, different databases,
different sources, such as Cloud or on-campus,
or potentially Excel files versus something
that's in a database and numerous others,
which we'll talk about today.
And the idea is to bring these together
and be able to look at them in a unified way.
We're here to talk about the FME platform.
It's made by a company named Safe Software.
Safe's mission is to help you maximize the value of
your data and Plaid is here to help you on that mission.
So one of the challenges we encounter a lot
in higher education is data silos.
For example, you may have the institutional research team
using a particular kind of technology,
the facility's team using a different one,
and the finance team doing a different one.
Often these data are stored in different places.
They don't talk to each other particularly well.
They're often referred to as silos.
By using data integration,
we can better address the challenges that silos pose.
Data integration allows us to bring together data
from these multiple different data sources as shown
and empower all of our users
across the different departments
to be able to do the kind of analysis they need
with the information they need.
Of course, we want to layer data governance on top of this
so that we have the right permission structure
for the right people to access the right data to.
Safe was founded a little over 25 years ago
and partners with more than 10,000 organizations worldwide
in 128 different countries.
There are 150 partners, including Plaid,
helping Safe deliver on this mission.
There are three primary products built
by Safe running under the FME brand.
FME, for those curious,
stands for Feature Manipulation Engine.
FME Desktop, which we covered in our first two webinars,
focuses on how to build and run workflows.
FME Server focuses more on how to automate those
and send different triggers.
We'll look at this today.
And FME Cloud is essentially the same thing
as FME Server, but it's hosted for you.
So there's none of the IT and administrative work required.
FME has the best support for spatial data
and is an outstanding data integration tool.
We'll show you some of both of those things today.
Now, it's about a lot more than just formats and systems
and geometry and repeatable workflows.
The goal really is to connect different teams
by sharing information seamlessly and securely between them.
So FME connects to
nearly 500 different kinds of data sources,
far too many to name if I were truly honest,
but you can see some of them in the image there
to the left with some famous format,
such as Oracle and PostGIS, PDF, Excel files, et cetera.
You can then extend the capabilities with custom connections
to other services that aren't in the list of nearly 500.
And then the next part is enable
your business leaders to make decisions using data.
This slide is intended just to show
how FME has innovated since the early 1990s.
It began in the geographic information science space,
moved towards a computer aided drawing and tabular data,
eventually into databases, and now as we move more
towards 2020, we're seeing investments
in business intelligence, indoor mapping,
augmented reality and virtual reality,
and artificial intelligence and machine learning.
We expect the pace of innovation to continue.
FME is just getting started, so lots of excitement to come.
Once you've connected to your data
regardless of where it lives, you can then transform it.
The goal here is to make it so that
your data is useful no matter how you're trying to use it.
So FME allows you to modify your data such as restructuring,
filtering, and calculating different kinds of fields.
It allows you to validate.
So check to ensure that you have high quality data
and this can be automated too.
And it allows you to adjust your workflow
because things change over time.
It's hard to have a rigid set of code,
say 10,000 lines of PL/SQL running scripts
when things change on a daily basis.
FME makes this a much easier proposition
to keep on top of changing requirements.
A big focus of today is how to automate workflows.
So the basic idea is to trigger based on some event,
such as maybe a student applies to your college,
you then assemble the information,
you use a visual interface as shown on the left,
and then the big goal is
to eliminate the manual effort that goes into this.
We'll show you some examples of this to come.
FME is supported across a wide variety of operating systems,
all the ones that you've heard of before, no doubt.
And you can also run it in Docker or Kubernetes
or hosted in the Cloud as discussed.
There are a number of different licensing options.
We focus on education.
So we've got the education subscription here.
This is based essentially on the number of students that
you have at your institution or in your school district.
Education applies for both K-12 and post-secondary,
and we'll show you a slide on that coming up,
but there are also other subscriptions
around local government and airports and utilities
and a number of others If you're in that domain.
The idea to the subscription model is it gives
you unlimited access to FME Desktop at FME Server
on your campus or at your organization.
This means that you no longer have to scale up your pricing
when you get more people who wanna use data for their roles.
This in our mind is a huge advantage, one of the big reasons
why we think FME could be a great solution
for data integration for educational organizations.
There is also a perpetual license.
So this is more individual-based
and it's designed for smaller deployments.
Last but not least, there's the Cloud deployment,
FME Cloud option, which pays
based on your usage, essentially.
More info is available on Safe's website, including pricing
for those outside of Canada and the United States.
So as mentioned, FME has a education subscription.
The idea here is that you get,
essentially an all-you-can-eat model.
So you can use as much of FME
as you want, Desktop or Server.
It's an annual cost that you pay.
We've got a slide coming up on that.
And you can turn it on basically whenever you want.
So when a new institutional analyst gets hired, for example,
you can deploy FME right away to them
without having to get additional approvals.
On the top right here, it's essentially free
for other departments to adopt.
So maybe your statistics division has adopted it early,
or maybe your facilities division has,
but student services is coming on board a little bit later.
Well, you've got the licenses available for them,
they can get running whenever they're ready.
this introduces predictable budgeting.
So it's an annual cost.
It's based on the number of students that you have
and it saves you procurement headaches
because it's predictable
and it's a single purchase instead of multiples.
This is the price sheet as of September 10th or today.
It's in Canadian dollars.
But to give you a rough idea, this is
what the cost of the education subscription would look like
and based on the number of students that you have.
So today we're going to talk about FME Server
and the major goal to FME Server is to automate workflows.
This allows you to bring workspaces that you've created
in FME desktop onto the web and allow anyone within
your network to harness the data integration power of FME.
This allows you to run workflows on a schedule
or based on an event being triggered.
We'll show you examples of both of those today.
You can keep data up to date automatically
and you can get notifications to yourself
or any other stakeholder involved in the process.
The licensing for the FME Server is based on the number
of engines that are running, but note the caveat
that the education subscription is an all-you-can-eat model,
so that part doesn't apply quite as much
if you go down that road.
You can also choose to run your FME Server on-premise
or hosted in the Cloud, such as on AWS or Azure.
So the real goal of using something like FME Server is
to save yourself time and money, automate those data flows
that otherwise you'd be doing manually.
So here's a few common use cases before we get
into our examples, just to give you some sense of
what others are using FME server for.
So some of these use cases,
including automated data integration,
delivering applications,
providing self-service data integration,
sending emails notifications,
delivering self-service data access,
data integration, obviously,
and scale power for data integration.
So some really common use cases we see out in the wild,
almost every government
that has an open data portal is using FME
in the background to make that possible, as an example.
Airports use it as well,
and we'll show you some examples coming up.
Education institutions are big customers,
but we expect this space to grow as well,
which is part of what brings us here today of course.
So here's an example, the City of Coquitlam,
which is a suburb just outside of Vancouver, Canada,
uses FME to manage a self-service data portal.
This allows citizens to be able
to download different kinds of spatial data
about the city itself.
The idea here is that it speeds up
significantly the delivery process of information
that people need to, say,
build new houses or redesign a neighbourhood.
And they figure that this saves somewhere two and four hours
of manual processing per request.
So in a city this size that actually adds up
to an awful lot over the course of a year.
And we see the same kind of potential in education,
though, not quite for city planning necessarily,
but imagine all of those data requests that you get
that you have to go to the database and run a SQL query for,
or run a script and SPSS, maybe eventually output it
to Tableau and turn it into a cross tab
and send it to somebody.
All of these steps can be done for you with FME.
FME Cloud has mentioned is
just the hosted version of FME Server.
So Safe Software hosts this for you
in a number of different regions on AWS.
Plaid can also help manage it if you have the desire.
But the basic idea here is the hosted version saves you
on configuration, IT costs,
and having your own hardware to run it on.
The licensing for this one is a pay-as-you-go model,
it's based on how much you use a server.
So a couple of bits of terminology
that we'll go through briefly before we demo it live.
So we're gonna talk about triggers coming up.
This is a big part of FME Servers.
So trigger is essentially what makes a workspace run.
So we can do this on a number of different ways.
One is to schedule it.
So we could say run every day at four in the morning
or run every minute if we really wanted to.
We can also trigger events based
on directories of files changing.
So today we're gonna demonstrate
how that works with Amazon S3,
but the same story would be true for Dropbox
or local folders or FTP or a number of others as well.
We could also base it
on receiving a particular kind of email or a webhook,
such as a message from Slack.
External messages as well, so a number of different services
by Amazon and Microsoft are listed there.
Those can trigger a workspace as well.
And then internal message messages are
within the FME server.
We'll show you some of that today as well.
So triggers essentially ask a workspace to run.
Actions do the running part.
So they can run a workspace,
they can filter or log messages and events.
They can send emails, which we'll show a little bit later.
They can send other kinds of messages too
through a web service or through one of the Amazon
or Microsoft services or Slack, et cetera.
It can also upload a file,
which we'll be showing you with Amazon S3.
And it can notify itself
that some other action is happening.
Server apps essentially are a client-facing view
or a user-facing view for FME Servers to run workspaces.
So an example of this is open government.
If you say, I want to know
how many citizens live in a particular place?
You can go to a portal and get that information.
That's almost always an FME Server app.
You're allowed to fully customize this.
So you can add your own corporate logo, colours,
banners, instructions, et cetera.
It looks pretty good once you put all this information in.
Gallery apps are essentially a level up.
So they provide a landing page and shows you
that there's multiple workspaces
that are available that they can choose from.
We're now gonna get into our demonstrations for the day.
I will briefly explain what the first one is
and then I'm gonna turn it over
to my colleague, Pat, to show it live.
So the first one is we wanna have FME Server send an email
based on an event that happens in Amazon S3.
The event that we're talking about here today includes
either creating a file, modifying a file,
or deleting a file.
This is the rough workflow.
So on the left, we see somebody places a file
in what Amazon calls a bucket, in the middle,
FME is paying attention to that bucket,
and if a change is made, it's gonna notify something.
And on the right, in this case,
the notification is to send an email to the user.
So from here, I'm gonna turn it over to Pat,
who's going to show you this.
Over to you, Pat.
Patrick: Thank you.
I will just share my screen here.
What we're looking at here is our FME Cloud instance.
One of the nice things about FME Cloud is
that Safe provides the ability to run this
in multiple different locales around the world.
We're running this in a Canadian data centre,
but you can run it in the US or numerous other places
where they use Amazon Web Services.
So where Amazon Web Services has a data centre.
We're looking at automations and we're gonna start with
our Basic S3 Listener that we've already created.
Just take a quick look at this automation,
it's very simple.
We have a trigger based on Amazon S3
and when something happens to the particular bucket
that we're looking at, we're going
to send an email based on that.
In this case, I've got the S3 bucket open elsewhere
and I'm just going to upload a file, in this case,
a screenshot of my desktop,
which isn't particularly exciting,
but will show us the functionality here.
So we're just gonna wait a sec
as this uploads, fingers crossed.
And it is going to take FME a few seconds to process this,
but in a relatively short order,
we're going to get an email
that's going to show up on our inbox here pretty quickly.
Just because we don't know exactly how long that'll take
while we're waiting for that, let's look at a second
slightly more complicated demonstration.
So in this demonstration, what we're gonna do is
we're going to look at a couple
of different server applications that we have
that are going to allow our users
to interact with different workspaces that we've created.
In both of these cases, what these are allowing us
to find the closest schools.
Assuming that somebody has just moved to London, Ontario,
in Canada, we can look up based on open data sets
that London provides both for addresses that exist
within the city and then schools
that are within the city as well.
We can look up what the near schools are and provide,
in this case, we're just gonna use email again
to give that information to our users.
We've got the workspaces already in FME and
we can see here the different configuration that we can do.
In this case, we've turned off, Require Authentication.
So anybody can come along and use this
as long as they have the URL.
So we could easily link to this from our webpage
or an open data portal or really anywhere else
that our users might wanna access this.
We can set up which parameters we wanna use.
In this case, we have an email address
where the recipient will receive the results.
And then we have an address that we wanna look up.
And then we can do some customization, as Andrew mentioned.
So we can customize background colour, logos,
we can add additional banners, even a browser icon
if we want, some footer text, and more logos and banners.
Once this is all published,
what we end up with is a webpage that looks like this.
So in this particular case, we have customized the colour
and our logo, we haven't added any footer text
or anything along those lines.
But what we can see is a relatively straightforward way
for our users to provide the information that they need
in order to get the information out of this workspace.
So I can indicate
where I want the results to go and a particular address
within London that I want to look up.
With this done, I can then press Okay
and it'll tell me it's running.
And again, this will take a couple of seconds to run
and ultimately we'll end up with an email.
Now in the interest of the demonstration,
what we've actually done here is coupled an app
with a automation on the FME Server site.
So running that application is actually pretty simple
and what it does is creates an internal notification trigger
for FME Server.
And when FME Server gets a particular notification,
in this case, on our webinar demo address topic,
it'll run another workspace.
If that workspace is successful,
it'll send an email to the email address provided,
and if it fails in any way, shape or form,
then that'll show up in our log.
And we can get much more complicated here.
We can start stacking more actions
if we wanted to log in various places or we run it
to run multiple workspaces in sequence,
we can set that all up as well.
Now, okay, so our first one, we just uploaded a file to S3.
We got an email here that indicated that there was an change
in S3 and our screenshot file got uploaded
and we can see that it was created.
This particular trigger will also run
if a file is deleted or if a file is updated.
So we can see here very quickly
what kinds of actions were taken.
In terms of the school,
we put in 2373 Main Street in London, Ontario,
and we picked out a couple of different kinds of schools.
So both elementary and secondary school,
and then English and French immersion.
So we found the nearest school of each of those types
to 2373 Main Street and we fired that off in an email.
And that all took just a couple of seconds to run.
Now, one of the things that you'll have noticed
about this particular example is that the
address had to be entered in a very particular format.
In particular, there was no periods or anything
along those lines, and that's simply
because of the way that London formats it's open data.
We wanted to make this a little bit
more foolproof, a little bit easier.
We can actually have our users select somewhere on a map.
In this case, we want to just allow them
to select a point in London, Ontario.
And we've got a little a button here
that's provided, and we can allow them to drop a point
anywhere in London and utilize that as the location
to find the nearest schools to rather than needing
to format a particular address exactly.
So in this, I'm gonna hit Confirm
and we're gonna run that as well.
Now this is a separate workspace
basically set up the same way.
We've got an FME Server topic, a workspace that runs
to actually go through the data and have a look
and then sends an email off to our end user.
And we've formatted the emails slightly differently.
In this case, because we've got coordinates,
we add those coordinates, we also find the nearest address
to those coordinates before we then find nearest schools.
So this is one of the ways that
we can put together multiple different features
of FME Server, in this case, apps and notifications
to all work together to provide a
very seamless experience to our end users.
With that, I am going to turn this back to Andrew,
who will take us through the last example.
Andrew: Thanks very much, Pat.
I wanted to chime in one other tiny bit on
that workspace that Pat ran.
It's pulling directly
from the City of London's open data as well.
So FME has a connector to make this possible essentially.
So we told it where to find the data and it pulled
all the right information based on the input
that the user selected on the map
or typed in by address as appropriate.
Okay.
So our next example is focused
on assembling data files together.
So our case example for this is a data set
that's called BC HEADset.
And so this is a data public data set
from British Columbia, Canada,
the six research-intensive universities.
So this demonstration intends
to reimagine the data submission process.
Just for your knowledge,
BC HEADset currently looks like this.
So it's a website and it has a number of Excel files,
different forms of understanding enrollment and students.
You can download the Excel files
and ultimately see pivot tables that look like this
and you can filter and so on that way.
So that's kinda what we're trying to replicate.
So for the purposes of this example,
I've taken the total enrollment workbook,
which is what you can see here
and taken it apart and rebuilt it.
So I should note that the demonstration is not affiliated
with the BC HEADset.
There we go.
Okay, so the current process roughly
based on an outsider's perspective, I know there's
some nuance to this that the people
that assemble the data set would add.
But the basic idea is
that there's six participating universities.
They ultimately provide a total
of seven institutional files.
The difference here is
that the province's largest university,
the University of British Columbia, has a major campus
in both Vancouver and the Okanagan,
which I'll show you on a map if you're not from BC,
those submit separately.
So there's actually seven submissions from six institutions.
Those go to an individual based
at the University of British Columbia,
who assembles them on behalf of the BC HEADset.
They make some required changes.
They do some transformations to the data
and this ultimately results in a combined Excel file of say,
total enrollment, like our example with a pivot table.
They then post it to the BC heads-up website.
But as you'll recall from earlier,
we want to automate our workflows.
So here we trigger, assemble, and eliminate a manual work.
Our re-imagined process looks something like this.
So we still have six participating institutions.
They're still gonna submit files,
but they're gonna submit them directly to Amazon S3
From there, FME is going to keep an eye on S3 for us
and it's going to notify say,
the institutional researcher as appropriate,
but it's gonna do all the heavy lifting for them.
It's gonna do the automated checks,
it's gonna combine the files,
and rather than just posting them to a single website,
it's going to post them to the web, it's going to post them
to Google sheets and in turn update a Tableau Workbook,
and it's gonna post them, the Excel files, to Amazon S3.
So that's where we're headed
From here, we can look at our S3 bucket.
What we can see is that we have a number of files already.
In the interest of moving the example along quickly,
I have said that a single institution has not
yet submitted their file.
So in this case, it's SFU, Simon Fraser University,
a mine and Pat's former employer.
And knowing the people involved with this,
they wouldn't be late.
So it's a funny example for those who have the inside joke.
But SFU is the one file that we're missing.
So I'm gonna upload that first.
So imagine somebody from SFU goes
to the Dropbox, hits Upload,
and then I'm going to drag the SFU file onto the screen.
It's identified this, and then I can hit Next,
and as appropriate set some permissions,
but we've already defaulted this to the intended permissions
for this example, and I'm going to hit Upload.
And it's nice to see the success message coming in
because my internet has been a bit wonky of late.
Okay.
So as mentioned before, we had a listener on S3.
So Pat had added a screenshot previously,
the listener has now told us that that has been deleted.
So we've set the listener to pay attention
about every one minute as an example.
And let me show you what that looks like
as an automation, just while it runs.
so on FME Server, if I go to manage automations
and I look for the BC HEADset S3 Listener,
what I can see here is a slightly more complicated workflow
than the one that Pat set before, but it's not very hard.
So what we've got is we've got the same monitor
for Amazon S3, and then we have some filters.
And the idea here is it's checking
for what kind of change was made to S3.
So if it's a create, we send a particular email message.
The basic idea here is we're sending a different message
for whether it's a Create, a Delete, or Modify.
So that's what the blue blocks do is they send emails
based on what happened with those filters.
And then down here at the bottom right is a trigger
that will tell a particular workspace on FME Server to run.
So we've got BC HEADset 1,
which is focused on receiving files.
So assuming that we've either created or modified a file
on S3, we're gonna run this particular workspace.
So let's take a look at that workspace
and I'll talk briefly about what it does,
and then we'll see what the automations have done.
So looking at the BC HEADset 1 workspace,
let's make that a little bigger so you guys can see it too.
Oops.
Okay, moving all the way to the left.
So the standard workflow in FME is that
we have a data source first, so a reader or a creator.
So reader might be something like we're gonna read
from Excel or we're gonna read from a database
or a GIS file, et cetera.
A creator is a way of kinda skipping this step,
where we create one or many dummy rows worth of data.
And then we have some transformers
that do a bunch of things to our data.
And then at the far end, we have a writer.
So reader or creator, then transformer, then writer.
So what we're doing first is we're checking Amazon S3
to see if there are any files on S3.
That's what our S3 connector does.
We then test just to throw away the folder
for the assembled file because we don't need it.
We then do some counting to check for
how many actual files are on the server.
And we could get a little more advanced on this
if we wanted to check the presence
of these very specific files that we had.
And then down at the bottom here,
we've also got a data source that says what files we expect.
So we're just comparing the two.
So we're saying we expect there to be seven files
and we're checking for whether that's true.
As we move to the right here,
we're then just combining those data sets.
So feature joiner and feature merger do
practically the same thing, they're just different ways
of joining together different data.
And we're basically keeping only the expected files
and then we're checking the combination of
how many files are expected versus actual files
and proceeding past this tester here
only if all the files are present.
We then put our data into a JSON template,
JSON is JavaScript Object Notation,
and then we notify FME Server.
And so this is an internal notification to the server
and the idea here is if it managed to pass this test,
as in all the files exist, then and only then do we go on
to the second BC HEADset workspace?
So that's our first workspace and what should have happened.
So let's take a look at our demo email
and it looks like it's caught up now.
So first it said, there's a new file
that's been added from SFU.
And then the remaining emails you see about assembled means
we've now got all the required data set
and I've built you the Excel file that you requested.
So let's take a look at the remaining pieces of this.
If I go back to view my workspaces,
or sorry, manage my workspaces,
and I view the BC HEADset to dataset,
sorry, view BC HEADset 2 in the visual form,
what I can see here on the left is
that it's taking JSON as an input.
So this was essentially the output
from my workspace 1, where I said,
"Only proceed if I have all seven files that are required."
We then do some footwork
around creating temporary directories
on the server for all of these files.
So the one on the left is just so that we have a container
for everything once we combine them, and the one
in the middle there is for each individual files.
So they serve slightly different purposes,
but do almost the same thing.
As we move towards the right, we connect to S3 again.
And rather than just checking for the presence of files,
we're actually gonna download the seven specific files
that workspace number one told us were present
and were the required files.
So we're gonna download one from each institution.
Once we've downloaded it, we can then read our local version
and combine them into one file.
And then we do a little bit more footwork
around updating the path name so that it works for Amazon.
So the next two steps here over towards the right are
we create a local Excel file, so that if we wanted
to publish it to our BC HEADset website, we could do that.
And then the last step on the right
there is publishing the updated file,
or sorry, the combined file to Amazon S3.
We've also done one other thing down here at the bottom,
which is publishing to Google Sheets.
So the reason we did this for a live demo is so that
our attendees can see a Tableau Workbook
that's been populated by this workspace.
So those all happen simultaneously essentially
and without any input from me, they just worked,
which is always a bonus in a live demo.
All right, so the end result of that is
we get a notification to say
that our file has been assembled and it appears
that I'm getting a few extra notifications.
So we'll see how this goes in terms of the next steps,
but in any case, we do have a file in our Amazon bucket,
which we can verify through the browser
by clicking on Assembled
and seeing that this was published at 10:41 a.m.,
which was about 30 seconds ago,
Pacific Time in North America.
So that's a great start.
The other thing we can do is
go back to Google Sheets,
which appears to frozen for me.
Okay, so I'll come back to that.
All right, so
here we go, okay.
So Google Sheets has now updated.
So if I look at my assembled total enrolled file,
maybe, there we go, we can see that our last update was made
just seconds ago by Plaid Consulting,
which is FME Server has done this work for me.
So I now have a whole bunch of data
and I've got update time, which has been shortened just
to the 10th of September to keep things simple,
and some other data sources about locations
and then justification, which I'll show you
in a second in Tableau, those are static.
But the good news is
our Google Sheet has updated as intended.
Okay, so from here, I promised you
that this would end up in Tableau rather than
on a static website with pivot tables.
So let's how that turned out for me.
Okay.
So on Tableau Public, and
let me put this in the chat for you.
Pat, we'll post that in the chat for you.
Okay, so right now we have a very boring
and grey Tableau Public workbook.
The idea here is that I wanted to show you it mostly live.
So remember how Google Sheets just got populated back
when this was last updated at 8:44 a.m. today,
there was no data to be able to show this.
So Tableau Public can connect directly to Google Sheets.
So we've updated Google sheets automatically using FME.
Tableau Public will update a few times a day,
but not usually on a perfect schedule for a live demo.
So I'm gonna hit Request Update
and we will give that a moment.
It shouldn't take too long though.
It's a pretty small data set.
Okay, here we go.
Almost.
Okay.
Momentarily, we'll have a Tableau Workbook.
You can see from my caching,
the hint of what it should look like.
All right, just double check.
Okay, so we're now updated as of 10:44 a.m.,
which is right now.
So now we can see that rather than just a grey box,
we actually have some content.
So if you're following along live,
the link is now in the chat.
Feel free to interact with this on your own as well.
There's an info icon at the top right that tells you more
about BC HEADset and what the purpose of this demo was
as well as links to the inspiration for this post.
So Wendy Shijia posted on the Flerlage Twins website about
how to draw curves on a map, which you'll see in a moment.
And also Adam McCann has a KPI dashboard,
which is linked there,
and it's the technique that you'll see in a minute.
It's probably the easiest way to explain that.
So I'm gonna hide that
and now I'm gonna follow the instructions.
So step one is click on the province of BC.
It's the one that's colour coded in black.
You can click anywhere.
Once we've done that, Tableau is gonna update a parameter
and allow me to view more information.
So this is the curves on a map.
I did this to test my skill
in Tableau just because I could.
So we were using SFU as our demonstrator earlier,
so let's pick on them again.
So once I click on SFU,
it loads some more information further down.
So the Adam McCann technique was just this one,
how to get the coloured bar to appear next to the text.
So what we can see is that enrollment over time
at Simon Fraser University has increased substantially
between when it started submitting to BC HEADset in 2005
and the most recent year, which I believe was 2018/19.
Yes, 2018/19.
So we can see that this has changed over time.
And if we scroll down a little bit further in on this,
we can see a bar chart down at the bottom
that helps us understand the top five disciplines
at Simon Fraser University.
If we pick a different institution
such as Thompson Rivers University,
our chart then updates and shows
that TRU has also seen a big change over time.
But that the top discipline list is a little bit different
at this institution.
If we pick one of the newer institutions
to start submitting,
we can see the University of British Columbia, Okanagan,
started its very first year reporting 56 students.
So UBC Okanagan, for those who don't know the story,
used to be part of the university college,
sorry, Okanagan University College.
It was split into two institutions,
one of which is UBC Okanagan,
one of which was Okanagan College.
So some students went one way, some students went the other,
depending on the program that
they were in and their personal preference.
But what we can see is
that odds are there's 56 was reported in the summer.
So just before the start of an academic year,
it didn't include the major semesters.
But it gives us nice text for a 6411% growth rate,
which is kinda fun, but the disciplines weren't reported.
So we have that.
If we look at the provinces smallest institution,
we see a different kind of trend.
So
in this case, enrollment has actually declined
in the years that they've been submitting.
But we can see that the disciplines are quite different.
So nursing is the most popular discipline based on
this data set in the most recent year as an example.
So that's basically what our intended process was.
So if I go back to PowerPoint here,
remember I said, each institution would submit to S3,
and in this case we said, SFU was the last one to submit.
FME is gonna do some automation for us,
it's gonna combine the files,
it's gonna do some quality checking,
which we didn't build for the demo,
but we would encourage because through building this demo,
we realized there were some mistakes in the data
that we cleaned up to make the demo work.
So we're gonna let those that submitted know
about this as well, just so that
they can fix their part if they need to.
FME then combines the files, notified me by email
that the process was going and if there were any errors,
which there were not, it then posted it to the web,
posted it to Google Sheets and to S3 as an Excel file,
and Tableau updated relatively automatically,
it'll do it overnight for sure with the latest data.
So this would save considerable time in the process
and make it so that our institutional analysis,
who's providing great value about understanding
why is this data changing, what does it mean,
and what are some of our options for moving forward,
can spend more time on that than collecting data
and massaging it together.
All right, so that's the end of the examples.
I wanting to mention that Safe Software has tons and tons
of free resources available at safe.com.
So you can get a free trial of FME
or you can get a free license for home use.
And because all of you work in education, note
that there are education licenses for teaching and learning.
So students and professors,
it doesn't work quite the same on the administration side,
hence the education subscription's a better option there.
There's also free online training and tutorials
and webinars, all of which I would highly encourage
because I never would have been confident enough
to deliver this session if it weren't
for having learned some of those things first.
And there's a great knowledge centre as well,
an active community that helps you be successful.
So some next steps, we are happy to do a demonstration
that's more specific to your organization.
Don't hesitate to reach out.
You can contact us at info@plaid.is
or through our website plaid.is.
Certainly, we'd be happy to work with you to come up
with a use case that might help your decision makers say,
"Actually this might be worth a serious look."
We'd highly encourage it.
As noted at the beginning, this webinar has been recorded.
It will be available on our website shortly
once we get the transcription finished.
You can contact myself or Pat through info@plaid.is.
Okay, and thank you so much for attending our webinar.
I will stop the recording here and turn it over
for questions if any of our attendees have any.