Imperatively Declarative


This past weekend, I did a [Twitch session for Manning](https://twitch.tv/manningpublications) in which I built a little serverless app with [Pulumi](https://pulumi.com) and [Mapbox](https://mapbox.com) and deployed it on [Amazon Web Services](https://aws.amazon.com). It was the first time I’d ever done such a thing — I’ve done many demos, but never a public live-coding session — and surprisingly (to me, at least), I actually enjoyed it. In a little over an hour, I wrote an app with Pulumi and TypeScript that renders GPS-tagged photos on a map with AWS [S3](https://docs.aws.amazon.com/s3/), [Lambda](https://docs.aws.amazon.com/lambda/), [DynamoDB](https://docs.aws.amazon.com/dynamodb/) and [API Gateway](https://docs.aws.amazon.com/apigateway/), and aside from a few little bumps, and my crazy-long COVID hair, everything worked out pretty well.

The hardest part was definitely setting up [OBS Studio](https://obsproject.com/), but now that that’s done, I plan to start doing more of these, as they’re (a) fun and (b) probably the best way to demonstrate how Pulumi really works and what makes it so awesome.

You’ll find the [full recording here](https://www.twitch.tv/videos/793943710), and the annotated code of the finished product’s [up on GitHub](https://github.com/cnunciato/photomap). Enjoy!


Live Coding with Pulumi, Mapbox, and Leaflet


Good news, friends! I’m writing a book.

In *Pulumi in Action*, I’ll be writing about how to use [Pulumi](https://pulumi.com), the open-source project I work on full time, to build cloud applications and infrastructure with [TypeScript](https://www.typescriptlang.org/) and [Amazon Web Services](https://aws.amazon.com/). I’m incredibly excited about this book — it’s my first — so I’m writing this post as a public expression of that excitement. (My wife often says I don’t often emote as much as I should, so I’m attaching an appropriately celebratory emoji as well, lest we be left with any lingering doubt. 🎉)

As of now, I’ve only written the [first few chapters](https://livebook.manning.com/book/pulumi-in-action), but since the book is the first ever focused on Pulumi, [Manning](https://manning.com) (the publisher) wanted to release it as a work-in-progress through [MEAP](https://www.manning.com/meap-program), its early-access program. Thanks to them, you can both read and purchase — or if you like, read and *not* purchase — the book today, and as new chapters roll out, at around once a month, you’ll get access to them immediately, in all of the finest digital formats of the day. It’ll cover a lot of surface area — infrastructure as code, Pulumi itself, cloud-native software development and delivery patterns, and more — and is generally aimed at helping full-stack developers get a solid foothold into the cloud. You can read the first chapter for free [on manning.com](https://www.manning.com/books/pulumi-in-action?a_aid=chris\&a_bid=20ca8d8a) today, and browse all of the examples [on GitHub](https://github.com/pulumi-in-action). The print version is scheduled to ship sometime early next year, and should come in at around 300 pages.

So that’s it — that’s the post. I *had* written around two thousand words recounting the tale of how it was that I, a mere web developer, came to write a book about [infrastructure-as-code](https://en.wikipedia.org/wiki/Infrastructure_as_code), but my wife, upon reading those words, declared that most of them were unnecessary: what mattered, she pointed out, was what was *in* the book. So I’ll save the full backstory for the eventual preface, and for now, leave you with this: if you’re a full-stack developer, you should check out [Pulumi](https://pulumi.com), and when you’re ready to learn more, [I’ll be here](https://www.manning.com/books/pulumi-in-action?a_aid=chris\&a_bid=20ca8d8a).

> **Update**: I’m still writing a book — just not *this* book. After the better part of a year, I decided to part ways with Manning — long (and rather odd) story — and [publish the book on my own](https://leanpub.com/pulumi). You’ll find the *new* (and still in-progress!) book on Leanpub. [Check it out](https://leanpub.com/pulumi)!


Pulumi in Action is available now in early access.


I’m writing a book!


I’ve been a user of [Adobe Lightroom](https://www.adobe.com/products/photoshop-lightroom.html) for several years now, and I’ve generally been happy with it. While I’ve never been in love with the Lightroom interface — it works, but design-wise it’s a bit on the clunky side, and dated, and it often performs sluggishy even on my newish Macbook Pro — but it’s generally gotten the job done, and there aren’t, even in 2018, many photo-management apps that can handle tens of thousands of photos and videos.

I’m not the only one in my house who takes pictures anymore, either. My wife is a great photographer herself (she has a much better eye than I do, actually), and my kids are starting to get into it, too. So managing everyone’s photos, and making sure we all have access to them, has been a serious challenge for me over the years. I’ve thrown together many an awful hack, some involving external drives, some shared network drives, synchronization schemes, DropBox (nope), even dedicated computers, but nothing ever seemed to work, mainly because Lightroom operates as a window onto your filesystem: you can do just about anything you want with it, but your pictures have to be *on your computer* (or on a drive connected to it), which makes sharing your library with your scrapbooking wife an invariably awkward and frustrating process.

So when Adobe announced Lightroom CC, a cloud-based approach and ground-up rewrite of what’s now called Lightroom Classic, I was excited: it looked like we might finally be able to keep all of our photos (and videos!) in the Adobe cloud, and share them all easily between our Macs and our iPhones. I downloaded the app, fired it up, chose Migrate Lightroom Catalog, pointed it at my `.lrcat`, and let it run.

![](/media/images/migrate-catalog.jpg)

The import took a while, and when that was done, it took another couple of weeks to get everything uploaded and synchronized. But the result was *glorious*: not only was my entire library available on my laptop, but it was also synced magically to my phone, to the Lightroom website, and *to my wife’s laptop and phone*, too — I was thrilled. And since the Lightroom CC mobile app lets you choose to have all of your photos uploaded automatically, whatever my wife happened to shoot with her iPhone would show up moments later on my desktop, and vice-versa. As a software engineer who understands the complexity of this problem, I was (and remain) thoroughly impressed with Lightroom CC. It’s not perfect, and it’s certainly not free, but it’s *by far* the most sophisticated and complete solution out there today.

There was one little problem, though.

When the migration from Classic to CC had finished, I noticed a number of my pictures (some 9,500 of them, in fact) had been dropped into a bucket labeled **No Date** — which was odd, considering that hadn’t been the case in Lightroom Classic, and irritating, given how date-oriented I am in my photography. (A bit of Googling led me to realize this is a [fairly common problem](https://forums.adobe.com/thread/2438359).) The only remedy seemed to be tweaking the Capture Date manually, one photo at a time, using the controls in the UI — which would’ve been fine, if there were only a few, but with close to ten thousand of them, I’d be at it for the better part of a week fixing them up, which seemed insane. There had to be a better way.

![](/media/images/lightroom-edit-date.jpg)

One of the nifty things about Lightroom Classic is that it uses a [SQLite](https://www.sqlite.org/index.html) database to store everything about your photos, including their dates, which means, if you’re a geek like me (I *love* SQLite), you can get in there and run queries to fix these kinds of problems on your own. Lightroom CC *appears* to use SQLite also, but even if I could update its database manually (the schema is completely different), a number of the photos in the dateless batch really did seem to lack one or more [EXIF](https://en.wikipedia.org/wiki/Exif) dates; the right thing to do would be to update the files themselves and set their date fields properly.

So I came up with a plan, and with a little trial and error, it worked.  Here’s what I did:

1.  First, I exported all of the photos in the **No Date** bucket into a folder on my desktop called `Exports`. The important thing here wasn’t the photos themselves, but their filenames. You’ll see why next.

2.  Then I wrote a little program to locate each of those files, using their filenames, in their original (i.e., Lightroom Classic-managed) locations, which for me happened to be on an external drive. The program would copy them into another folder on my desktop called `Originals`, leaving me with two folders, side-by-side: one called `Exports`, the other called `Originals`.

3.  After checking to make sure the number of originals matched the number of exports — meaning I had an original for every No Date export — I deleted the contents of the No Date bucket in Lightroom CC.

4.  Then I ran another program called `exiftool` on each file in the `Originals` folder, setting its EXIF capture date using the original file’s creation date. This might not work for everyone, but given my workflow, it happened to be a safe bet that the creation date would be pretty close to the actual capture date, so I went with it.

5.  Finally, with all of my `Original`s now properly dated, I imported them into Lightroom CC.

That was it — no more No Dates!

I’ll run through the steps in detail below, in case you happen to find yourself in this predicament, too. You’ll have to do a little [Bash](https://www.gnu.org/software/bash/) scripting to make this work, so I’m assuming you’re on a Mac (apologies to Windows users) and that you’re relatively comfortable on the command line.

Okay! Let’s make this work.

***

## Step 1: Create Two Folders on Your Desktop

Pretty straightforward. Again, I named mine `Exports` and `Originals`.

## Step 2: Export Your Dateless Photos

In Lightroom CC, click **By Date**, then scroll to the **No Date** folder and click it. Choose Edit » Select All, then File » Save To, then browse to your `Exports` folder and click Choose. Let the process finish — it’ll probably take a few minutes.

## Step 3: Find Your Originals

Open a Terminal and make a new file on your desktop called `find-originals.sh` and make it executable:

```shell
$ cd ~/Desktop
$ touch find-originals.sh
$ chmod +x find-originals.sh
```

Then copy the following script into that file and save it:

```bash
for f in $1/*; do
  exported="$(basename $f)"

  # Locate the original
  original="$(find $2 -name ${exported})"

  # If the original isn't found, log it and continue;
  # otherwise, copy it over
  if [ -z "$original" ]; then
    echo "Could not find a match for $exported"
  else
    echo "Found a match for $exported -- copying"
    cp -vf "$original" $3/
  fi
done
```

The script accepts three arguments:

*   the path to your `Exports` folder
*   the path to your Lightroom Classic originals
*   the path to your `Originals` folder

As I mentioned, I had exported my **No Date** files into a folder on my desktop called `Exports`. My originals were located on an external drive called `Elements`, and the empty destination folder, `Originals`, was on my desktop alongside `Exports`. So the command I ran was:

```shell
$ ./find-originals.sh \
    /Users/cnunciato/Desktop/Exports \
    /Volumes/Elements/Media/Lightroom \
    /Users/cnunciato/Desktop/Originals
```

When that finally finished, I had about 9,500 files in `Originals`, which matched the number of images in the **No Date** bucket in Lightroom CC. I could now safely delete the bad ones.

Feel free to comment out the actual `cp` statement at first, just to make sure you’ve wired things up correctly. When you’re ready, run it for real. (We’re just copying, here, so don’t worry — all you have to lose is a little time and disk space, temporarily.)

## Step 4: Kill All No-Dates

You can do it! After all, you still have your Lightroom Classic originals — and now copies of them, even — and you’ve verified the number of files in `Originals` matches the count of your **No Date**s in Lightroom CC. So go on — select all, Edit » Delete, and get ‘em outta there.

Well done.

## Step 6: Write Out Some EXIF Dates

As I said, I was sure a good number of my photos really didn’t have EXIF dates (over a thousand of them were from an old Samsung flip-phone), and I wanted to fix that problem properly by modifying the files themselves, rather than just their Lightroom metadata.

![](/media/images/me/flip-phone-selfie.jpg)

There’s an [amazing little command-line app](https://sno.phy.queensu.ca/~phil/exiftool/) called `exiftool` that’ll both tell you which files are lacking certain EXIF fields (e.g., dates) and also let you set their values, either “in place” (meaning on the source file itself) or, if you’re a little paranoid like me, on a copy of the file, preserving your original.

So I did this in two steps: first, I ran `exiftool` to find out how many dateless photos I had, and then I ran it for real to copy each file and burn a capture date into it. (I shoot a lot of video, too, and `exiftool` handles video formats, also — even the obscure ones from the early 2000s! It’s really great stuff.)

So assuming you’ve still got that same Terminal open on your desktop, run `exiftool` on your `Originals` folder to get a count of how many dateless images you’re dealing with:

```shell
$ exiftool -createdate -if 'not $datetimeoriginal or ($datetimeoriginal eq "0000:00:00 00:00:00")' ./Originals
```

This command asks `exiftool` to tell you which of the files in `./Originals` is missing a `datetimeoriginal` value, which is what Lightroom considers a capture date. When you run it, you’ll get some output for each file, along with an aggregate result at the end. If, for example, I’d had three files in my `Originals` folder, two with dates and one without, my output might’ve looked something like this:

```shell
$ exiftool -createdate -if 'not $datetimeoriginal or ($datetimeoriginal eq "0000:00:00 00:00:00")' ./Originals
======== ./Originals/IMG_2949.PNG
    1 directories scanned
    2 files failed condition
    1 image files read
```

Personally I find this output a little inscrutable, but I believe what it means is that two of those files *have dates* (i.e., they failed the `-if` condition of `not $datetimeoriginal`). If instead we’d gotten something like this:

```shell
1 directories scanned
3 files failed condition
0 image files read
```

… then we’d be done, as `exiftool` would be telling us that all of our files actually *have* capture dates. In my case, though, I had around 3,000 without them, so I decided to go ahead and update them all.

To do that yourself, run:

```shell
exiftool -P -O ./Output/ '-datetimeoriginal<filemodifydate' -if '(not $datetimeoriginal or ($datetimeoriginal eq "0000:00:00 00:00:00"))' .
```

This is essentially the same command you ran above, but with some additional options instructing `exiftool` to *do some stuff:*

*   `-P` means “preserve my originals”
*   `-O ./Output/` means “put the modified copies into the `./Output/` directory”
*   `'-datetimeoriginal<filemodifydate'` means “set the capture date using the date the file was last modified” (which for me was also the date it was created)

If you’d like to use a different algorithm for setting your own EXIF capture dates (or even just learn more about what this awesome little program can do for you), [check out the `exiftool` docs](https://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html).

Once this was finished — it ran surprisingly quickly, considering how many photos I was processing — I imported a few into Lightroom CC just to make sure things looked right. When they did, I imported them all.

## Step 7: Re-Import!

The hard part’s done. Now, just choose File » Add Photos, browse to your newly populated `Output` folder, and click Review for Import. If things look good, click that shiny Add Photos button, then sit back and watch the magic happen.

## Step 8: Celebrate

Well done! With any luck, once Step 7 completes, that **No Date** folder should be a thing of the past. If that’s the case, go ahead and delete everything we created from your Desktop. You’re done.

![Cheers.](s3/images/20140622-14-56-18-4.jpg)

***

I’m sure I’ll write more about Lightroom in general; while I’m definitely a CC convert now, I still use Classic to manage my kids’ catalogs (they each have their own — they shoot way too many pictures of absolutely nothing), and the CC software is continually evolving, which is awesome (and yes, occasionally frustrating). Nothing is perfect, but man, I have to say — on the whole, glitches and all, I’m still really impressed, and stoked that we’ve finally found system that works pretty much seamlessly for us.


How I managed to migrate from Lightroom to Lightroom CC


Bad Dates


First post! Exciting, right?

It’s really more like my five-thousandth, if you count everything I’ve stuffed into various micro- and regular-sized websites over the years (so many, and nearly all lost to history, alas), but that probably only matters to me, anyway. You’re here because — well, I’m not sure why you’re here. But I’m glad you are nonetheless.

I don’t have much to share yet, but I will. For now, I’ll just say thanks for stopping by, and encourage you to return: I plan to take lots of pictures, make lots of little movies, and write virtual reams (reams, I say!) of words, some of which you might actually enjoy, if we’re lucky.

In the meantime, here’s a snapshot from a hike we went on recently, at the beautiful [Redmond Watershed Preserve](https://www.wta.org/go-hiking/hikes/redmond-watershed-preserve).

![A mossy fern.](/media/images/mossyfern.jpg)

Have a lovely day, and do come back soon, won’t you.


Imperatively Declarative

How (and why) Pulumi is different.

Live Coding with Pulumi, Mapbox, and Leaflet

I made a thing with a map! It was fun.

I’m writing a book!

Pulumi in Action is available now in early access.

Bad Dates

How I managed to migrate from Lightroom to Lightroom CC

Hello Again, World!

It’s the first post! Kind of.