Adding time-travel to embedded Google Street View containers
I've been working with the Google Maps Javascript API recently at work. We're developing a remote building surveying app to find good candidates for panelized deep energy retrofits. Our users are shown an interactive streetview and satellite map so they can inspect a building of interest and collect data on it.
At a user testing session, someone requested a time travel feature, "like in the real google maps" which would allow them to see the evolution of a building over time, or get a better look e.g. in the winter when folliage won't block the view.
I was happy to oblige, thinking I could simply add the misleadingly named imageDateControl option to the configuration. I quickly realized that it apparently only adds the date the imagery was taken at the bottom of the container.
After some digging, it turns out that time travel is not offered by the API as of August 2023. A pending feature request exists since 2015, but gives no updates or timelines - perhaps Google engineers could consider working more than 1h a day. Interestingly though, and luckily for me, all of the necessary data is discreetly returned by the API since at least v3.51, and I was able to hack the feature together over a few days.
Final Result
Image 1: Final result of the time travel feature in streetview
My time travel supports:
- Switching between the available time periods at a location.
- Maintaining the selected time period (or the closest available one) while moving around.
- Maintaining the selected time period (or the closest available one) after moving the Pegman, which is actually more than the real google maps!
Here's the code: https://github.com/lhovon/google-maps-time-travel/tree/main
Note: you'll need a Maps API key (there's a $200/month free credit).
Basic Setup
For these example, we'll start with a simple setup with a streetview and satellite map container side by side. The timetravel dropdown can be implemented with only the streetview, but obivously we'll need the map to move the pegman around in the second part of this post.
You can find the starter code here: https://github.com/lhovon/google-maps-time-travel/tree/starter-code
Data
Calls to StreetViewService.getPanorama() return an undocumented array called time with all the available panoramas at a location and their date. The name of the date field is not stable and varies across API versions (as well as within a version IIRC). This data has been returned at least since November 2021, based on this issue tracker comment.
{
"copyright": "© 2023 Google",
"imageDate": "2015-09", // Current pano date
"location": {...},
"links": [...],
"tiles": {...},
"disabled": false,
"Hv": {...},
"su": [],
"takeDownUrl": ...,
"time": [ // Time travel panoramas
{
"pano": "Yr5v-08rMV4JuITgQTPM8g",
"Qm": "2011-09-01T04:00:00.000Z"
},
{
"pano": "MEggMtiMLy6Un_OsX__kiA",
"Qm": "2015-09-01T04:00:00.000Z"
},
{
"pano": "sEhNhchuNFMPBoJfIlk2Rg",
"Qm": "2016-06-01T04:00:00.000Z"
},
...
]
}
The game plan is to:
- Get the panoramas from the API
- Create a dropdown element containing them
- Change the panorama by calling StreetViewPanorama.setPano() when the selected dropdown option changes
Development Part 1 - The Dropdown
Let's first make function to generate the dropdown options from the list of panoramas.
The function also takes a target date. We'll use it to select the option whose date is closest to the target. For now, there will always be a panorama for the target date, but later with the Pegman that won't be the case.
function generateTimeTravelOptions(panoArray, targetDate) {
const options = [];
// Convert the selected date string in YYYY-mm format to a Date
const dateSplit = targetDate.split("-");
const selectedPanoDate = new Date(dateSplit[0], parseInt(dateSplit[1]) - 1, 1);
let closestPanoEl;
let minDiff = Infinity;
// Assuming the list entries have 2 keys: "pano" and the variably-named date key
const dateKey = Object.keys(panoArray[0]).filter(e => {return e !== "pano";})[0];
// Iterate through the available times in reverse
// order so the most recent date appears at the top
panoArray.reverse().forEach(el => {
const date = el[dateKey];
const option = document.createElement("option");
// the option's value is the panorama ID
option.value = option.id = el["pano"];
// User visible text of the dropdown option
option.innerText = date.toLocaleDateString("en-US", {
year: "numeric",
month: "long",
});
// Keep track of the smallest absolute difference btw
// the selected date and all the available dates
const diff = Math.abs(selectedPanoDate - date);
if (diff < minDiff) {
minDiff = diff;
closestPanoEl = option;
}
options.push(option);
});
// Set the minimum difference element to selected
closestPanoEl.selected = true;
return options;
}
Create the dropdown in the HTML, note the onchange function that sets the streetview panorama to the option value.
<div id="streetview">
<div id="time-travel-container">
<label id='time-travel-label'>Time Travel
<select id="time-travel-select" onchange="window.sv.setPano(this.value)"></select>
</label>
</div>
</div>
Let's add some styles for the dropdown.
#time-travel-container {
display: none;
background-color: #222222;
opacity: .8;
position: absolute;
right: 0px;
z-index: 10;
margin-top: 10px;
font-family: Roboto,Arial,sans-serif;
border-bottom-left-radius: 2px;
border-top-left-radius: 2px;
padding: 6px 8px;
}
#time-travel-label {
color: #ffffff;
font-weight: bold;
font-size: 12px;
}
#time-travel-select {
display: block;
height: 25px;
margin-top: 5px;
}
Finally, we'll call generateTimeTravelOptions after setting up the map and streetview, append the options to the dropdown and make it visible. Making the dropdown visible before the streetview and map are initialized, results in it visibly moving into place, which is no bueno.
async function findPanorama(svService, panoRequest, coordinates) {
...
// Send a request to the panorama service
svService.getPanorama(panoRequest, function (data, status) {
if (status === StreetViewStatus.OK) {
...
// Generate the list of options, with the initial panorama date selected
const options = generateTimeTravelOptions(
data.time, data.imageDate
);
// Attach the options to the select
document.getElementById("time-travel-select").append(...options);
// Make the dropdown visible
document.getElementById("time-travel-container").style.display = "flex";
}
...
});
}
This works at a fixed location in the streetview, but the list of options is not updated when we change locations. For that, we need to write a few more lines.
Selecting an option in the dropdown triggers the pano_changed event. We'll catch that event and query the StreetViewService for all the other panoramas available at the new position. We can then fill the dropdown with these new options.
I also noticed that using the links (the white arrows in streetview) tends to trigger 2 pano_changed events, so to avoid doing an unecessary network call to StreetViewService on the second event, we'll check that the new panorama is not equal to the previous one and return if it is.
// Save a ref to the current panorama
window.lastPanoId = data.location.pano;
sv.addListener("pano_changed", () => {
const newPanoId = sv.getPano();
// Skip duplicate events
if (window.lastPanoId === newPanoId) {
console.debug(`Extra event on ${window.lastPanoId}, returning!`);
return;
}
// Get the available panoramas from StreetViewService as before
svService.getPanorama({ pano: newPanoId }, (data, status) => {
if (status === StreetViewStatus.OK) {
const options = generateTimeTravelOptions(data.time, data.imageDate);
// replace the previous options with the new ones
document.getElementById("time-travel-select")
.replaceChildren(...options);
}}
);
});
This should work! You should now be able to move around in the streetview using the links or by clicking around and maintain your chosen time period, or the closest available one at your location. You can find the code for this functionality here: https://github.com/lhovon/google-maps-time-travel/tree/dropdown-only. Note that it you click to move very far away from your current position, it may not successfully maintain the time period, but for moving around a building, for example, it works well.
Development Part 2 - The Pegman
With our current implementation, moving the pegman will break the time travel and cause the streetview to reset to the default date at that location (not always the most recent). This is actually the same behaviour as in the official Google Maps implementation, but we can go further. This could be useful to our users e.g. to quickly teleport to the other side of a building of interest.
New game plan:
- Detect a pegman drag-and-drop
- Get the panorama that's closest in time to the previous panorama
- Change the streetview to that panorama
Detecting pegman drag-and-drop
Ideally, the map's event system would have a pegman_dropped event of some sort, but of course this isn't the case. In fact an Aug 24 2023 reply on this IssueTracker thread indicates that it will not be implemented. So once again, we'll have to hack something together ourselves.
Comment #9 in that same thread proposes a working solution by listening to events on the map:
[Deleted User]<[Deleted User]> #9 (Feb 28, 2017 12:03PM)
(...) In short if you have had a mousedown event (on the div), haven't had a mouseup event (on the div), are currently getting mousemove events (on the div), and are not getting google maps dragged events (on the map object) then you have pagman.
My solution is similar but attaches events directly to the pegman by way of a MutationObserver. The above solution might be more efficient, as you'll see that the MutationObserver is constantly re-attaching events as the pegman element gets regenerated after being dropped.
By inspecting Mr. Pegman, we see that it can be document.querySelector'ed with img[src='https://maps.gstatic.com/mapfiles/transparent.png']. However, we can't do that before it's been instantiated in the map, which seems to occur after the map idle event.
Instead of being notified, we can use the MutationObserver API to intercept the pegman's instantiation and any subsequent regeneration.
// Register a mutation observer to attach events to the pegman
const observer = new MutationObserver(attachEventsToPegman);
const config = { attributes: true, subtree: true };
observer.observe(document.getElementById("satellite"), config);
function attachEventsToPegman(mutationList) {
for (const mutation of mutationList) {
if (mutation.target.getAttribute("src") ===
"https://maps.gstatic.com/mapfiles/transparent.png")
{
// Without this, we easily get hundreds of listeners attached
if (mutation.target.getAttribute('listenersSet')) return;
console.debug('Attaching events on pegman');
mutation.target.addEventListener("mousedown", () => {
console.debug("mousedown");
window.pegmanDropped = false;
window.pegmanMousedown = true;
});
mutation.target.addEventListener("mouseup", () => {
console.debug("mouseup");
if (window.pegmanMousedown) {
window.pegmanMousedown = false;
window.pegmanDropped = true;
}
});
mutation.target.setAttribute('listenersSet', 'true');
}
}
}
You might be tempted to disconnect the observer once the first events are attached, but that won't work. As I mentionned above, the pegman is apparently regenerated after each drag-and-drop, so we need to keep attaching listeners to it, else they will only fire once. Setting the listenersSet attrbute allows us to avoid attaching hundreds of extra listeners while the pegman is dragged around, which triggers mutations.
Image 2: Without a countermeasure, we end up attaching hundreds of useless listeners as the pegman is dragged around
At this point, we can detect when a drag-and-drop has occured through the window.pegmanDropped variable. Let's use this to update the panorama after a drag-and-drop.
We'll want to log the last panorama's date to know which date to choose among the available panoramas at the new location.
window.lastPanoId = data.location.pano;
// Add this after our other window variables
window.lastPanoDate = data.imageDate;
In the pano_changed event listener, we'll check if a pegman drag-and-drop has occurred and if the current panorama date is different from the last. If that's the case, we'll fetch the available panoramas at the new location and update the streetview to the one that is closest in time to the last panorama.
To do this, we can modify the generateTimeTravelOptions function to return the closest panorama's ID and date in addition to the options. (You'll have to update the calls to it as well to now receive an object.)
function generateTimeTravelOptions(panoArray, targetDate) {
...
let minDiff = Infinity;
let closestPanoEl, closestPanoDate;
...
panoArray.reverse().forEach(el => {
...
if (diff < minDiff) {
...
// Keep track of the date (used just for logging)
closestPanoDate = `${date.toISOString().split('T')[0].substring(0, 7)}`;
}
});
// Return the closest pano ID and date
return {
options: options,
closestPanoId: closestPanoEl.value,
closestPanoDate: closestPanoDate
};
}
By calling generateTimeTravelOptions with the last panorama's date at the new location, we'll know which panorama to change to.
We have to be careful when changing the panorama because it will trigger another pano_changed event, and we can easily get stuck in an infinite loop of events. To make matters worse, the map already fires 2 pano_changed events after a pegman drag-and-drop (for no apparent reason)!
What I ended up doing is setting another window variable to mark that we need to change the panorama and that panorama's ID. Then, on the second pano_changed event, we detect that we should change the panorama and trigger a custom event, whose handler actually changes the panorama. This triggers a third pano_changed, which proceeds normally as we're past the drag-and-drop, and the new and last panorama's dates are equal anyway.
Overall, it looks like this:
// Custom event launched when pegman is dropped and we need to change panos
sv.addListener("pano_change_needed", () => {
// The sleep is needed, otherwise it breaks for some unknown reason
sleep(0).then(() => {
sv.setPano(window.shouldBePano);
console.debug(`Changed pano to ${window.shouldBePano}`);
});
});
// Sleep function used above
function sleep(time) {
return new Promise((resolve) => setTimeout(resolve, time));
}
sv.addListener("pano_changed", () => {
console.debug("pano_changed");
const newPanoId = sv.getPano();
// Detect when we need to change the panorama after a pegman drop
if (window.panoChangeNeeded) {
window.panoChangeNeeded = false;
console.debug(`Starting change to ${window.shouldBePano}`);
// trigger custom event
return event.trigger(sv, "pano_change_needed");
}
// Skip duplicate events
if (window.lastPanoId === newPanoId) {
console.debug(`Extra event on ${window.lastPanoId}, returning!`);
return;
}
// Get all available panoramas at the new location
svService.getPanorama({ pano: newPanoId }, (data, status) => {
if (status === StreetViewStatus.OK) {
console.debug(`New pano: ${newPanoId} - (${data.imageDate})`);
console.debug(`Last pano: ${window.lastPanoId} - (${window.lastPanoDate})`);
// If the pegman was just dropped and the new panorama's date is not equal
// to the last panorama's date, we manually change the panorama to the
// one closest in time to the pre-pegman drop date.
if (window.pegmanDropped && data.imageDate !== window.lastPanoDate) {
window.pegmanDropped = false;
console.debug("Pegman dropped and new pano date not equal to last."
);
// Get the ID of the panorama closest in time to the last date
const { closestPanoId, closestPanoDate } =
generateTimeTravelOptions(
data.time,
window.lastPanoDate
);
console.debug(
`Will change to closest pano ${closestPanoId} (${closestPanoDate})`
);
// Set this variable so we know we need to change the pano
window.panoChangeNeeded = true;
window.shouldBePano = closestPanoId;
return;
}
console.debug(`Generating dropdown options for new pano`);
const { options } = generateTimeTravelOptions(
data.time,
data.imageDate
);
document.getElementById("time-travel-select")
.replaceChildren(...options);
// save the current pano date for next time
window.lastPanoDate = data.imageDate;
window.lastPanoId = newPanoId;
}}
);
});
Unfortunately, we see the panoramas changing which is not the smoothest user experience, but that's as far as I could go.
Overall, the solution is very hacky. It was developed by digging around and experimenting, which has led to abominations such as the random but necessary sleep(0) before responding to the pano_change_needed event. It will also most likely break for some future version.
Hopefully it gets rendered obselete by an official implementation, which may or may not be in the works (see bonus notes below)!
It's very possible that I over-complicated this and missed something obvious, but AFAICT I'm the first to post an implementation for this online. Please reach out to me if you have a better solution, I'd love to hear about it!
Bonus Notes
Hints of a future implementation
While digging around the maps and streetview code, I found a few interesting things. Once of them being hidden placeholders for a possible future implementation of time travel (or most likely access to the existing feautre in Maps). You can reveal them in the HTML (I'm using API version 3.53) by switching their display (underlined in red in the pic).
Image 3: Hidden time period select in the HTML returned by Google
The associated jsaction (underlined in blue) is found in the JS, we see an empty function.
Image 4: The timeline.show's jsaction is not currently implemented
Alternative dropdown placements
I tried to add the dropdown to the existing container with the streetname etc., but it turns out the Maps SDK really doesn't like you playing around with the elements inside. Sometimes even adding attributes to elements will cause a total crash.