Request a free site audit

Interested in offering ad free site memberships?

Life news: I got a dog

Categories

POUND hack, take two, rebuilt with AWS services

Over the weekend I refactored my POUND project into services and put it on AWS. Repo is private for now, but I wanted to explain the architecture.

  1. The client, which handles a few jobs:
    • Fingerprint the browser
    • Prepare a pageview data object
    • Update the hash
    • Send the pageview

    83kb in compiled form, could probably get it sub-5okb by dropping the cookie library. Most of the page weight is in the fingerprinting. This site is currently carrying the client (you should see a hash in the URL above).

  2. AWS API Gateway. I ended up making this using a GET request because they are faster. I am passing my pageview data through headers. Using GET should also mean that the browser silently retries requests. API Gateway provides configuration for CORS and also comes with SSL standard.
  3. AWS Lambda function that takes the pageview event, does some light transformation/enrichment, and then puts a new item into DynamoDB.
  4. AWS DynamoDB table. Previously I had been using Firebase Storage but this approach should be more scalable. I flirted briefly with AWS Firehose into S3 & Redshift, but there was too much configuration required and I like the flexibility of NoSQL for now.
  5. Create React App/Vis.js. This part is pretty much the same as before, just moved to S3 and refactored for the new data supply. New link.

The visualization needs the most work right now. I hit the limits of Vis.js and really need to move into d3. The timeseries nature of data is missing currently, and I need to figure out a good way of conveying the “multiple pages” concept (or maybe it’ll be better to isolate to a single GUID at once). I’m also collecting user agent and IP address now for removing bots and geolocating later.

Did you enjoy this post?

Signup to receive a weekly email containing my new posts, curated links, and book reviews.

Thank you for subscribing.

Something went wrong.

Comments

Follow me on Twitter
Thoughts

Lenfest Institute and Digital First Media?

What if Alden let another organization manage its newspaper assets as a blind trust? It would lose a great deal of flexibility in using DFM assets to leverage other companies it owns, but it would be able to wash its hands of the growing public relations crisis. Furthermore, it would give space for the strategic direction of DFM to be explored and pursued without the added baggage of hedge fund cross-percolation.

Notes on dynamic meters

Notes on newsletters

Notes on the membership model for news

Penn State Campuses Lose University Recognition After Risk Analysis Review

Essays

Capturing Shawmont Station before its $1,000,000 preservation begins – the oldest extant passenger rail station in America

Originally a 18′ by 36′ stone house (Wissahickon Schist), the structure wouldn’t have stood out from the other country homes in this part of Philadelphia, at the tip of the Manayunk Reach, situated at the end of today’s Manayunk Canal Towpath.

Testing WordPress Gutenberg on a high volume news site

Water, sand, and societal change

Apple releases MapKit JS just as Google Maps pricing about to change

Starting July 16, the cost of using Google Maps at scale will increase substantially, so perhaps it’s no coincidence that Apple quietly introduced MapKit JS amidst last week’s Worldwide Developers Conference. It’s very possible that I’m missing a key assumption or two in this modeling, but if the pricing differences are even close to the scenario presented above, we are witnessing a tectonic shift in maps as service.

EverQuote and patent medicine

In a thread begun October 2016, Washington Post technology director Aram Zucker-Scharff tweeted about the shady advertising practices of EverQuote, a Boston-based startup. Since then these ads have become prolific on the web (and nearly as prolific are Aram’s tweets documenting the malfeasance).

Lenfest Institute and Digital First Media?

What if Alden let another organization manage its newspaper assets as a blind trust? It would lose a great deal of flexibility in using DFM assets to leverage other companies it owns, but it would be able to wash its hands of the growing public relations crisis. Furthermore, it would give space for the strategic direction of DFM to be explored and pursued without the added baggage of hedge fund cross-percolation.

Receive a weekly email with newly posted content

  • About one email per week
  • Includes original posts, curated links, and book reviews

Thank you for subscribing.

Something went wrong.

Send this to a friend