This site you created is a working example of a WordPress powered aggregation hub- a means for a site to automatically pull in blog posts (and other content) participants in a connected course are publishing in their own space.
It makes use of the Feed WordPress plugin to do much of this aggregation work. It is very powerful… As one super hero learned…
There are a lot of moving parts…
This page of instructions is long! And this is the minimum to get going…
Figuring our RSS feeds can get complex. The Feed WordPress plugin sometimes does unexpected things. The way syndicated content interacts with a theme (e.g. featured images) can cause quirks.
That said, for the most part, for what it does; it’s almost magical. In over 5 years of activity, on the DS106 site it was aggregated over 60,000 blog posts from 20+ different course iterations and an open community. The Thought Vectors in Concept Space hub at VCU brought together syndicated content from 6 different sections of a course, plus from open participants. The use of categories enabled slices of this flow by course section, by students, by faculty, by open a participants.
Feed WordPress 101
In support of the 2014 DML sponsored Connected Courses “course” Alan Levine (technically also writing this post), created a five part “how to” series on setting up one of these sites.
There is quite a lot of information there; but this template installed at StateU.org has done a good chunk of that set up for you. But it is still worth using for reference (these posts are also syndicated in this demo site, aren’t we clever?):
- Basic Concepts of Syndication – and what to think about even before you touch that WordPress thing
- Installing and Setting up Feed WordPress – Minimal settings, and planning the way content is sliced, diced, and recombined
- Feeding the Machine – How to get RSS feeds into the aggregator without losing a finger
- Some Feed Magic – Optional ways to improve feeds from sites such as flickr, twitter, etc, creating a twitter archive, RSS Feed TLC
- A Few More Tricks – leveraging categories, adding attribution, setting featured images
These are some things I ponder before even touching WordPress. There is even more from Feed WordPress 101 including a bit more explanation of the syndicate concept.
- Where will I ask participants to publish? In a wide open course, you probably cannot dictate what blog platform they use, so you will end up dealing with all kinds of crazy choices like Google sites and Weebly (Weebly is very BAD with RSS, sorry Weebly lovers. It’s cute, but limited). Platforms that work well with this set up include WordPress (free or self-hosted), Blogger, tumblr, Squarespace, and Medium.
If you can scope this to say WordPress.com and/or Blogger, your life will be a bit less crazy with tech support. For Project Community, in 2012-2014, we had students create tumblr sites; in 2015 we had them do group WordPress.com blogs.
- Blogs devoted to Your Course are easier. It is alot alot alot easier to syndicate all posts from a blog than dealing with tags and categories.
- Where will commenting happen? The typical set up is that the aggregation hub displays the title of the posts and an excerpt; all links go out to the participants blogs. In this case comments will happen “out there”. If you want comments to happen in the course hub, you will want to change the settings so all links are “local” (Feed WordPress makes a full copy of all syndicated posts even if you link out, so it is always archiving)
- Will you want to sub group blogs? Are there groupings that will help to separate syndicated posts by say course sections? groups? roles? If there is a value in doing this, decide on categories up front. When you add a blog to your hub, you can tell Feed WordPress to put all syndicated posts into these categories as needed.
Setting Up Your Own Hub Site
Once you are working with your copy of this site, here are some first things to do. Let’s make your own categories to organize blogs.
In your WordPress Dashboard, go to
Categories. It will look like this:
You are not required to use this structure; it is an approach that has worked well for other connected courses.
- Course Info Use this category for posts you write on the site as announcements or other information.
- Featured Once you start syndicating posts, it’s a good idea to add a category like this for posts worth highlighting. You can make these available as a link, as a menu item, and in this blog, as a front page display. Consider this an approach to curating the best posts out of what might be hundreds that come into your hub. You could make this a task for other students to do (?)
- Miscellaneous The default category. Every time a WordPress site leaves this as “Uncategorized”, a cat somewhere will cry.
- Syndicated Okay this is a category we will add to every blog you susbscribe to, it provides a way to see the full flow.
- The rest of the categories are “child” ones of syndicated, these are ones you might use to group syndicated posts. Maybe it is be different sections, different courses, project groups, etc. You can delete all of the silly “Cowboy” ones, and insert your own. Or do not use any, for a small class, just the Syndicated category may be enough.
So go customize your categories.
Normally on setting up a site, you would have to set all of the default settings for Feed WordPress (under the
Syndication link in the WordPress Dashboard). But we have done all of that for you. Still, you may want to review them as well as the Feed WordPress 101 post on the setup.
The important things to understand now are:
- These settings are the default for all sites you add to the hub. You can override every setting at the per subscribed blog level.
- These settings will take any subscribed blog posts use of tags, categories, labels and convert them all to tags on the hub. This makes all of the aggregated used methods of categorizing posts the same. Thsi way, we can use the hub’s Categories to organize the whole site.
Adding Feeds to the Site
In this demo site, and likely for a course-sized hub, it’s easier to add blog urls directly to the Feed WordPress plugin. Larger, wide open courses can be set up with a form for participants to add their sites directly (I had dreams of making this into a plugin, but alas, time). How you get your participants blogs (email? LMS) is up to you.
A middle ground solution is setting up a google form to collect participant information. I did this for ETMOOC, and processed over 500 submitted feeds.
For the purposes of this workshop, I made a form so we can share each others StateU (or any other) blog URLs. The idea is to get a collection of URLs you can experiment with. So first complete the form:
When done you should have access to the responses (or go directly to the spreadsheet).
If you would like to make a blog URL request Google Form, I made a generalized version of one that you can copy to your own Google account.
So what do you do with the URLs?
For full details on adding feeds, see Feed WordPress 101 on Feeding the Machine.
Let’s add a site to our hub. The easiest ones to add are sites where the entire blog is what we want to subscribe to (all posts). Feed WordPress will be able to identity the site’s RSS feed automatically.
When you go to
Syndication in the WordPress dashboard, you will see all blogs it is subscribed too; a few of them come with this copy of the site. We will deal with removing these links soon. But you add new subscriptions by pasting in the top right field for “New Source”.
Make sure it is a full URL (starts with “http://”) and that it actually points to the public URL for the blog (sometimes participants will send you a URL that is their private editors view). It has to be a URL that shows the blog when the owner is not logged into it.
Feed WordPress does some work to find the blog’s RSS feed- there might be anywhere from 1-6 that it finds; usually it is the first one. If there is a preview of a post in the right box, the feed is good.
You should get a confirmation message that the feed was added. We have one more step- to associate the blog with a sub category that you set up earlier. You can click the
Configure Settings link here and then click
Categories from the top settings link, or from any feed in the list, hover over and click the settings like for
Scroll down to the part of the settings with the default Categories used for this feed:
The default settings will add all posts that come into to the site to the Syndicated category. But we can additional sub categories for this blog, say to add all posts syndicated from it to the “Cowgirls” and “Town Folk” categories.
This is how you can create different views of your course’s syndicated content. Note that any changes will apply only to newly syndicated posts. If you do this at some point after you have been syndicating posts, you will have to manually add categories. Getting this organized ahead of time will save you grey hair.
We have set this blog up like one being set up; it will not automatically check sites until we change a setting. But you can manually check all subscribed sites from the list of Syndicated Sites by clicking the blue
Update button at the top of the screen.
You can also click the
Update Now button to the right of a single feed to check just that site.
In a few moments, Feed WordPress will look for new posts, and if there are, it will create copies of them as new posts on your site.
So how do you automate this? Under
Syndication in your dashboard go to
Feeds & Updates. Under Feeds and Scheduling change the setting from
cron job or manual updates to automatically check for updates after pages load:
What this means is that visits to your site by anyone will keep it updated. Do not worry, it does not check every site on every page load. Feed WordPress is smart. Savvy. It keeps track of sites in needs to update, and will only bother ones that it has not checked for at least one hour.
This is important to know, because when students publish a new post, it will take up to an hour before your hub will syndicate it.
Cleaning Out Feeds
You may not want to have the demo feeds that came with this blog. Feed WordPress has some handy options for dealing with feeds you may no longer want to have in the site. When you need to delete or turn off a feed, go to Syndication in the dashboard to see your feeds, hover over the feed you want to edit, and click
You have useful powerful, great responsibility options here!
- Turn off the subscription for this syndicated link Use this o stop looking for new posts. You might use it at the end of the semester, or just when you know you do not want to get new content. You can always re-activate it later.
- Delete this syndicated link and all the posts that were syndicated from it Use this to get rid of the subscriptions and content for blogs you do not want (like the feeds this blog came with). This is the Big Housecleaner Option.
- Delete this syndicated link, but keep posts that were syndicated from it (as if they were authored locally). Now this is very powerful. Your students may end up deleting their blogs after a class, or put the blog to other use. This option will stop checking the blog for new posts, and any links in your site that previously went to the external site, will automatically link to a local archived version on your site.
Fixing Bad Feeds
Any feeds that Feed WordPress cannot communicate will show up as yellow in your list. There is a “bad feed” in your site now:
The problem is that the blog site entered
https://www.tumblr.com/blog/re-new-media-art is not the public URL or its feed (this is the link a tumblr blog owner copies from their logged in dashboard). The actually URL for this blog is https://re-new-media-art.tumblr.com/
Your exercise is to see if you can use a Feed WordPress action to fix the feed.
Feeds that show up yellow will take some detective work to fix. Often the problem is when you try using a blog’s category or tag address to syndicate, when you are trying to get only the posts from a category. Feed WordPress cannot find their RSS feeds, so you have to manually figure it out.
See the Become a Feed Detective section in Feed WordPress 101 on Feeding the Machine
Working With This Theme
This site uses the free Gridbox magazine theme, chosen for its clean layout and use of featured images. A WordPress plugin takes care of converting the first image found n a post to be its featured image. If none is found, the “swirl” image is used (an exercise for you is to find where this is done ;-).
You might customize the menus. The front page uses a layout driven by widgets, that you can customize, re-arrange, add things to via
Widgets in the section labeled
Of you want to promote a post to the “Featured” area, just find it among the posts, and add
Featured as a category. The three most recent will show on the home page, but all can be found via the Featured category link.
Code has been added that the random link will redirect you to a randomly chosen Syndicated post. Another page generates a list of all blogs that are syndicated into your site (there must be one syndicated post from a source for it to show up).
This should be most of what you need to start building a working blog aggregator site. I have plans in the future to put all of this functionality into a plugin so you could turn any WordPress theme into an aggregation hub.
Once done with this post full of instructions, you can edit its Publication Status to
Draft so you can have it as a reference in your own site.
Featured Image: flickr photo by jonlclark https://flickr.com/photos/jon-clark/6287411573 shared under a Creative Commons (BY) license