Observations with Alecks

a different perspective

What is a podcast?

Whether you listen to them or not, it goes without saying you have some idea of what a podcast is. Maybe you make one yourself. Maybe you use a podcast app to listen to a few regularly. Above all else, you've likely heard something along the lines of “did you hear about that podcast?” It's part of our culture.

Yet, while there is general agreement that podcasts exist, is there agreement on the definition?

Podcast (noun):

An audio programme produced on a regular basis, delivered over the Internet in a compressed digital format and designed for playback on computers or portable digital audio players, such as the iPod. — WiktionaryInterestingly, users removed “RSS feed” from the definition in 2014

a program (as of music or talk) made available in digital format for automatic download over the Internet — Merriam-Webster

a digital audio or video file or recording, usually part of a themed series, that can be downloaded from a website to a media player or computer — Dictionary.com

A series of digital media files distributed over the internet to which a user can subscribe by means of a syndication application. — The American Heritage® Dictionary of the English Language, Fifth Edition

All of these touch on technically relevant descriptions of how a podcast is distributed or the medium in which it's distributed and still completely fail to describe the culturally disruptive phenomenon podcasts have become. When you casually converse about a podcast with someone chances are low that you're talking about the particularities how you listened to it — unless you're deeply involved in the surrounding ecosystem of podcast creation, hosting, or application development.

Even more, none of these definitions are in agreement. Sometimes podcasts are regularly produced, themed, audio, video, subscribed to, automatically downloaded, or... music?

Categories

Like anything else in the world, people like to assign podcasts to categories. The current state of the podcast category system is sort of like Yahoo's web directory, which was from the mid-1990s.

Some applications have come up with a few methods of search, but usually nothing is super reliable without a lot of curation. Otherwise you are either given a specific podcast name to look up, or a creator, having put a keyword on their podcast, might get lucky and have their podcast trending under the keyword they used. Maybe you're looking for something new and use the category to discover it during browsing.

Take music, for example. Listen Notes has a music category for music podcasts. It's great at first glance, but go through a few to find something new and you'll realize there's a problem.

The Is-About problem

Categories given on podcasts only tell you what the podcast is about, not what the podcast is. Browsing through the podcast “music” category, you will find podcasts both about music and podcasts that are music.

Tailored user experiences

When you set out to do an activity, you typically expect that activity to remain consistent until you're done with it. Music listening and podcast listening affect your brain differently, after all.

Mixing the two activities is relegated to specific types of playlists, like Spotify's automatically generated “Your Daily Drive.” When the content is mixed, you expect it.

Spotify is only able to do this by requiring creators to identify their content as music or podcasts up front. Even some existing podcast music experiences depend on specific RSS feeds.

No wonder it's confusing to define what a podcast is! The assumption is — unless you have the ability to specifically force people onto your platform and categorize your content appropriately — any RSS feed serving audio media is just a “podcast.” But, given the Is-About problem, we know that's not true.

If we had a way to identify music separately from podcasts in a decentralized manner, we would automatically provide a way to search for music without getting it mixed up in our podcasts. One might even develop an application specifically to listen to music served from RSS feeds.

Podcasting 2.0

How does this fit into Podcasting 2.0? Dave Jones, from Podcast Index, states the following:

Podcasting 2.0 is a set of forward looking ideas combined with the technology to realize them. It's a vision for what the podcast listener experience can and should be. That experience has stagnated for over a decade, with almost all of the improvements coming in isolated sections of the ecosystem. There hasn't been a single, unified vision from the podcasting community acting together with one voice. So, we've ended up with fragments of innovation across the podcasting landscape with no central driving goal in mind. Podcasting 2.0 is the expression of what that goal could be.

The original movement of “Podcasting” took the concepts of radio-oriented audio programs and disrupted an industry. While RSS has been used successfully for other things like blogs and news, efforts beyond radio-oriented audio have largely stagnated while the internet ecosystem moved forward.

Podcasting 2.0 is the second wave of this disruption. Documentarians can release new documentaries without worrying about censorship for promoting controversial ideas or lost revenue from simply being unglamorous. Video creators can work with hosts to distribute their content without covering it in unwanted, unrelated advertisements. Authors can get their work out to their audience without being stomped on by big tech. Independent musicians can take back control of their content and their income streams.

Your audience wants your content. They do not care how they get to it as long as it's a reasonable experience and will even reward you for your content if they find it valuable.

In Podcasting 2.0, this content is available to any application. Not just the ones the “platforms” allow you to use.

Source of truth

But, Podcasting? Some may be asking, what about [insert other standard or platform here]? I'm a big fan of ActivityPub, for example. Blockchain tech is very promising to solve otherwise hard global problems.

But the key differentiator is RSS XML has remained an open, valuable, and uncomplicated way to to define what content you have available to any application that wants to see it. Any person with a computer can resonably create an application that reads RSS.

The fact that RSS features have largely stagnated over the past decade or more isn't due to any fault of RSS itself — no one has stepped up nor had the momentum to unify the community. After all, doing things in the open for everyone requires a certain level of agreement on how to make new features available. It just so happens enough of us have decided we're not gonna take this anymore.

RSS remains a source of truth for content — not just podcasts — but we're missing a way to identify non-podcasts.

The Podcast Namespace

If you haven't kept up — or, perhaps, just haven't heard — part of the Podcasting 2.0 movement is to extend RSS to make new features available. Finally, after nearly two decades.

Given the Is-About problem identified above, one thing we need to add to The Podcast Namespace is a way to identify content outside of its category. I've written a basic proposal to get the process started, but we need much more discussion on such a critical idea.

It's prudent for us to identify content for the purpose of a user experience but also for the application. The fact is, some content will use more/different Podcast Namespace features than others. For example, some in our community are looking at how to tell an application that a feed is an audio tour.

An “audio tour” is a type of “podcast” that has location markers for a user to follow along while listening. The only way to identify if something is an audio tour right now is to, well, scan all of the millions of available podcasts to check or simply guess. This works but is unreasonable to any user or developer.

Developers

Particularly, we need developers to get engaged. We already know content creators exist for the content discussed, many of whom are struggling to get by working on their passion projects.

Once we can come to a conclusion about how to identify this content, applications can be created to use it. Podcast applications already exist, but using them to listen to anything outside of a “podcast” is a poor user experience. There's nothing inherently wrong with that, podcasts are the main thing RSS has been used for.

Music and audiobooks have gained a little more traction because they “fit” into existing applications, but mixing them into “podcasts” is often just messy. It's a different experience even if they are all just “audio.”

The concept isn't limited to audio, either. Video has been available as “vodcasts” for a long time, but it tends to be an afterthought in podcast applications. That's partially because there simply aren't a lot of video RSS feeds, and hosting video is more difficult than hosting audio; but there's also no standard way to identify if a feed is primarily for videos without guessing.

Some have already talked about how to create feeds for cooking recipes, too. This is a completely new concept. Imagine having a recipe application that can use content from other people anywhere in the world without you having to browse through a search engine and loading questionable advertisements... all while hoping the recipe is displayed in a reasonable manner. With value4value, you could even reward these chefs for their recipe from the app.

Telling the application what a feed is for is arguably more important than telling the user. The user either already knows what they are looking for or are looking for ways to discover something new. They don't care if your parmesan chicken recipe is distributed in the same way as their favorite comedy podcast.

Conclusion

Whether it's limited to a culturally significant idea of people talking about things sort of like an old-time radio show, or a method of distribution of digital media, we're not here to decide what “podcast” means.

Importantly, with Podcasting 2.0, we finally have a way forward to identify other types of content in a socially-relevant, meaningful way while allowing creators to control and distribute said content to their audience.

Right now, most of this content consists of typical podcasts. We are working together to change that. Let's stop waiting for others to do it for us.

#podcasting20 #rss #music #audio #video #bitcoin

RSS Delivery

No one likes polling.

Ever since the dawn of the open RSS ecosystem, applications poll RSS feeds. That's how it works, right? They check a feed for updates on some kind of schedule. Maybe, if they feel like tackling an unsolvable problem out of sheer curiosity, they add some learning algorithm to figure out when the feed actually changes to save resources.

Polling, most of which is done in regular intervals, is the most reliable method of knowing when a feed is updated within a decentralized ecosystem no one controls. Even today this is how most RSS feed aggregation runs because it's the only way to avoid missing anything. This isn't a problem in its own right, but it's inefficient.

It becomes an issue when:

  1. A feed gets extremely popular, which can easily cause unwanted bandwidth costs or even take down smaller operations.
  2. A single host manages a significant amount of RSS feeds. They add up!
  3. Time-sensitive content is released. Say one wants to publish a breaking news story or a live steam. Polling will catch it late, and increased polling frequencies are frowned upon (often rate-limited) due to the above issues.

Until now, users and services have often relied on other options for updates, particularly for time-sensitive content:

  1. Outside systems like social media have always offered a means to notify users of updates, but it's not a great user experience for anyone. This helps #3, but still requires polling. Often it causes more polling with the user refreshing rapidly.
  2. Centralized systems, usually implemented by podcast services as back-ends for client applications — especially on mobile — offer a way to hide polling from users. It helps #1 & #2, and works, but it's still polling and hiding this from users can cause confusion.
  3. Otherwise entirely closed systems with their own notifications (YouTube, Twitch ... the list goes on). These usually solve the issues for users, but it's no longer open! I'm not going to discuss the philosophical implications of closed systems, but they still get real time updates wrong, because real time updates are hard.

What about WebSub?

Yes, WebSub helps with some of the above issues. It has two main problems:

  1. WebSub attempts to move the problem from polling feeds to maintaining WebSub subscriptions. It's great in theory. In practice it's just another problem to solve and it's ultimately just another unreliable link in the chain. Applications frequently fall back to polling.
  2. WebSub requires a server to obtain updates. This isn't an issue for podcast applications that run their own services, but what about standalone applications as intended by the spirit of open RSS? What about the last mile? Standalone applications still end up polling, and especially get left out of time-sensitive content.

The Last Mile

The last mile is a relatively new problem within the scope of the open RSS ecosystem. Every application used to poll for content. Live streams, outside of obscure internet radio, didn't even exist or weren't widespread enough to be relevant.

Fast forward to today, most podcast applications run their own services and/or depend on large aggregators to do the heavy lifting. Many trusted Apple for this task, until they couldn't, and the Podcast Index only stepped up as an alternative because it's still such a difficult problem. Even more, the aggregators still have to be polled! They don't have the resources to become a notification system en masse.

Podping.cloud

Enter Podping.cloud:

Podping is a blockchain based global notification system for podcasting. Feed urls are written by the publisher to the blockchain within seconds of a new episode being published. Anyone can monitor for those updates and only pull a copy of that feed when it shows up on the chain.

Podping.cloud architecture

What does that mean? It consists of two parts:

  1. An on-boarding mechanism for hosts to migrate to with ease — something similar to WebSub but simpler to manage.
  2. A standardization of data interchange mechanisms — the “podping” namespace — to broadcast podcast updates onto the Hive* blockchain.

*Note: This is really just a specification of a JSON schema. Hive is an implementation detail.

Decentralized podcast updates!

Given Podping, a standardized way to communicate podcast updates into Hive, anyone can listen for them on the Hive blockchain. An important characteristic of what the Hive community has developed is a standardized HTTP API for applications to utilize, as opposed to having to download and manage the Hive blockchain themselves (though they could if they wanted to).

What does this look like for an application? Below are a few real transactions from the Hive blockchain:

{"_id": "cabd5cc591662b4cb131fb546ae9189d104a00ee",
 "block_num": 54014222,
 "id": "podping",
 "json": "{\"version\":\"0.2\",\"num_urls\":3,\"reason\":\"feed_update\",\"urls\":[\"https://feeds.buzzsprout.com/981862.rss\",\"https://feeds.buzzsprout.com/1722601.rss\",\"https://feeds.buzzsprout.com/1287197.rss\"]}",
 "required_auths": [],
 "required_posting_auths": ["hivehydra"],
 "timestamp": "2021-05-19T02:39:06Z",
 "trx_id": "ebd5b45772e119c38f75e246d4b70d60ee527716",
 "trx_num": 23,
 "type": "custom_json"}
{"_id": "6b7fe74973283bf99d76ef9b81ced193c43ebe84",
 "block_num": 54014229,
 "id": "podping",
 "json": "{\"version\":\"0.2\",\"num_urls\":2,\"reason\":\"feed_update\",\"urls\":[\"https://media.rss.com/njb/feed.xml\",\"https://feeds.buzzsprout.com/1749995.rss\"]}",
 "required_auths": [],
 "required_posting_auths": ["hivehydra"],
 "timestamp": "2021-05-19T02:39:27Z",
 "trx_id": "394ecb5449f6a7d9472432915ed2241628e0716e",
 "trx_num": 20,
 "type": "custom_json"}
{"_id": "c6ed805969a3c86634f34352e44a232e907336e1",
 "block_num": 54014236,
 "id": "podping",
 "json": "{\"version\":\"0.2\",\"num_urls\":1,\"reason\":\"feed_update\",\"urls\":[\"https://feeds.buzzsprout.com/1575751.rss\"]}",
 "required_auths": [],
 "required_posting_auths": ["hivehydra"],
 "timestamp": "2021-05-19T02:39:48Z",
 "trx_id": "b3a97ec7bf97604194755913953b28b6de21c403",
 "trx_num": 12,
 "type": "custom_json"}
 {"_id": "73c63d127e040049f0d6c2f118c1c00e46da17da",
 "allow_curation_rewards": True,
 "allow_votes": True,
 "author": "<redacted>",
 "block_num": 54015435,
 "extensions": [{"type": "comment_payout_beneficiaries",
                 "value": {"beneficiaries": [{"account": "hiveonboard",
                                              "weight": 100},
                                             {"account": "tipu",
                                              "weight": 100}]}}],
 "max_accepted_payout": {"amount": "1000000000",
                         "nai": "@@000000013",
                         "precision": 3},
 "percent_hbd": 10000,
 "permlink": "<redacted>",
 "timestamp": "2021-05-19T03:39:54Z",
 "trx_id": "7d4012b4b7e4901bbaf910a71166a78f5ac9b186",
 "trx_num": 22,
 "type": "comment_options"}
{"_id": "502ddccc9a1e4bb86b7379497e6b17e943c869d4",
 "block_num": 54014474,
 "id": "sm_find_match",
 "json": "{\"match_type\":\"Ranked\",\"app\":\"steemmonsters/0.7.89\"}",
 "required_auths": [],
 "required_posting_auths": ["nandito"],
 "timestamp": "2021-05-19T02:51:45Z",
 "trx_id": "b41cf36a88d798d15e31b1d2b718cd7fa4176b1b",
 "trx_num": 49,
 "type": "custom_json"}
{"_id": "1dc0e544c25978fcf868e40623e930d5bbdc97fa",
 "block_num": 54014474,
 "id": "sm_submit_team",
 "json": "{\"trx_id\":\"8a965173a51cf0457264196066febf3040d9c720\",\"team_hash\":\"946c2011bd2ec8844fcdc2bbf946a738\",\"summoner\":\"C1-49-SM0LILHETC\",\"monsters\":[\"C1-64-TPHW58AV34\",\"C1-50-1AMDZHJ7OG\",\"C1-47-S6HDP0EA5S\",\"C1-62-AQOTC401J4\",\"C1-46-1BP1BKEY3K\"],\"secret\":\"TZte3wNaM7\",\"app\":\"steemmonsters/0.7.24\"}",
 "required_auths": [],
 "required_posting_auths": ["postme"],
 "timestamp": "2021-05-19T02:51:45Z",
 "trx_id": "8ff17c754ff1d9621e9e0014439e720086ab5e85",
 "trx_num": 50,
 "type": "custom_json"}

A few interesting things to note here. It's nice to have all of the information, but it's a bit of a fire hose. How can we clean this up? We can hide it within our application, which might be acceptable for some servers... however it's still all unnecessary bandwidth. Particularly the last two non-Podping operations as well as the non-custom_json type.

Walking the last mile

In order for this to be efficient for client applications — especially mobile applications with limited resources — there's a couple things we need to address. Note: I am only familiar with the beem python library at this moment in time, so it's possible these two points are moot.

  1. Server side filtering of custom_json operations. Currently the python implementation, beem does this on the client side, hiding it from the user. It still pulls in every block even if you tell it to only look for the custom_json operation.
  2. Server side filtering of the custom_json id — even if we pulled in only custom_json operations, we're still getting other data from other applications. Really, we only care about id='podping'.

Given those two changes, we can size down the fire hose to the following podping operations:

{"_id": "cabd5cc591662b4cb131fb546ae9189d104a00ee",
 "block_num": 54014222,
 "id": "podping",
 "json": "{\"version\":\"0.2\",\"num_urls\":3,\"reason\":\"feed_update\",\"urls\":[\"https://feeds.buzzsprout.com/981862.rss\",\"https://feeds.buzzsprout.com/1722601.rss\",\"https://feeds.buzzsprout.com/1287197.rss\"]}",
 "required_auths": [],
 "required_posting_auths": ["hivehydra"],
 "timestamp": "2021-05-19T02:39:06Z",
 "trx_id": "ebd5b45772e119c38f75e246d4b70d60ee527716",
 "trx_num": 23,
 "type": "custom_json"}
{"_id": "6b7fe74973283bf99d76ef9b81ced193c43ebe84",
 "block_num": 54014229,
 "id": "podping",
 "json": "{\"version\":\"0.2\",\"num_urls\":2,\"reason\":\"feed_update\",\"urls\":[\"https://media.rss.com/njb/feed.xml\",\"https://feeds.buzzsprout.com/1749995.rss\"]}",
 "required_auths": [],
 "required_posting_auths": ["hivehydra"],
 "timestamp": "2021-05-19T02:39:27Z",
 "trx_id": "394ecb5449f6a7d9472432915ed2241628e0716e",
 "trx_num": 20,
 "type": "custom_json"}
{"_id": "c6ed805969a3c86634f34352e44a232e907336e1",
 "block_num": 54014236,
 "id": "podping",
 "json": "{\"version\":\"0.2\",\"num_urls\":1,\"reason\":\"feed_update\",\"urls\":[\"https://feeds.buzzsprout.com/1575751.rss\"]}",
 "required_auths": [],
 "required_posting_auths": ["hivehydra"],
 "timestamp": "2021-05-19T02:39:48Z",
 "trx_id": "b3a97ec7bf97604194755913953b28b6de21c403",
 "trx_num": 12,
 "type": "custom_json"}

This on its own is a huge improvement, but we can do more...

One step further

See the json attribute? That's our Podping schema! What if we only subscribe to one or two podcasts? We don't need the rest of the results.

Thankfully, there's a straightforward way to handle this: JMESPath

JMESPath is a query language for JSON.

Let's say we only want to get results from the podcast “Quality during Design” in the above list. We can test this in a quick way with the jp cli tool and the JMESPath contains function.

echo '{"version":"0.2","num_urls":3,"reason":"feed_update","urls":["https://feeds.buzzsprout.com/981862.rss","https://feeds.buzzsprout.com/1722601.rss","https://feeds.buzzsprout.com/1287197.rss"]}' \
| jp "contains(urls, 'https://feeds.buzzsprout.com/1722601.rss')"
true

Behold! We just defined a mechanism to query Podping json from a custom_json operation. In this example, the contains function being run by jp is returning true, telling us that this example indeed includes the feed URL we care about.

Why is this important? Couldn't we just parse the json object in our code and check for the URL?

Well, yes... but JMESPath provides us a way to run this query on the API server, without the API server requiring knowledge of our dataset. Meaning, given an implementation of this query on the Hive API, we would happily only get the results we need from the API.

If we only subscribed to one podcast we could get update notifications for it without having to parse through all new Hive blockchain events.

This is truly potential for a whole new generation of a decentralized notification system, and Podcasting 2.0 is driving it.

Hopefully some of the above ideas can be incorporated into the Hive ecosystem in a way that will benefit everyone.

#podcasting20 #podping #jmespath #rss #hive

Enter your email to subscribe to updates.