Capturing User Media Streams

The MediaDevices interface allows accessing a user's webcam and microphone, as well as allowing screen sharing. It requires a secure connection and prompts the user for permission after which the output - aka MediaStream - can be used within your application.

We'll specifically be learning how to use getUserMedia() to stream a user's webcam video and compose it with an SVG inside of HTML canvas to generate a screenshot, like mine!

There are a few essential parts of requesting a media stream, as in a user's video source like a webcam and/or an audio source like from a microphone.

The starting point is a call to navigator.mediaDevices.getUserMedia() along with an object holding the requested "constraints." The constraints include which types of media streams you would like - video and or audio - and other features like preferred dimensions. For the purposes of our holiday card, we're also going to request the front-facing camera as the preferred.

A Promise is returned from getUserMedia() along with the media stream if successful. The media stream can contain multiple "tracks" for each of audio and video sources.

We can wrap up the minimal required pieces as follows:

const handleSuccess = (stream) => {
  const video = document.querySelector("video");

  window.stream = stream;
  video.srcObject = stream;
};

const handleError = (error) => {
  console.error("Error: ", error);
};

const startWebcam = () => {
  navigator.mediaDevices
    .getUserMedia({ video: true })
    .then(handleSuccess)
    .catch(handleError);
};

In your HTML you would have an empty <video> element, and a <button> to trigger startWebcam(), such as:

<video autoplay muted playsinline></video>
<button onclick="startWebcam()">Start Webcam</button>

This is of course missing some important features, particularly the way to stop the webcam, but if you'd like you can test this in something like CodePen. If you haven't previously given your localhost or CodePen access to your webcam, your browser will display a prompt for you to give permission.

We also placed a few attributes on the video element which are critical to the streaming working cross-platform. The autoplay attribute prepares the video element so that as soon as it receives the stream it will play it back live. The muted and playsinline attributes are primarily to ensure the stream successfully plays back on iOS, as described in their video policies announcement.

1. Our initial display will position the SVG of the card frame illustration over the video element. Then, when a user selects "Start webcam", they'll see a preview of what their holiday card screenshot will look like.
2. Once they select "Take Photo", we'll stop the video and pass the video stream and SVG into HTML Canvas which has a method that we can use to set the composite as the src of the image.
3. The user can then save the screenshot like any other image on the web.
4. If they want to retake, they can again select "Start Webcam" and we'll reset the image src to the SVG frame.
5. Additionally, we'll flip the start action to stop so that they can cancel the video access without taking a screenshot.

Our generator has two sections: one to hold the card display, and one to hold the action buttons.

<div class="holiday-card">
  <video class="holiday-card__video" autoplay muted playsinline></video>
  <img class="holiday-card__img" src="/img/12DaysCardFrame.svg" alt="">
</div>
<div class="holiday-card__actions">
  <span id="holiday-webcam-instructions">Webcam required to enable taking a photo for the holiday card</span>
  <button type="button" class="holiday-card-webcam" aria-describedby="holiday-webcam-instructions" data-state="offline">Start Webcam</button>
  <button type="button" class="holiday-card-photo" disabled>Take Photo</button>
</div>

We've included a few features for accessibility by adding some instructions so that non-sighted users will get a bit more context for why there is a "Start Webcam" button. These users have a few modes of navigation including tabbing between interactive elements and scanning text content. In scan mode, they'll receive the instructions first due to the DOM order. In tabbing mode the button name will be read first, then the instructions due to the association created by aria-describedby.

Since "Take Photo" cannot perform any action until the webcam stream is available, we have set the disabled attribute.

On the webcam button, we've added the data-state attribute so we can keep track of whether the webcam is currently offline or streaming.

We'll be using CSS Grid and one of my favorite tricks to stack elements within a single grid cell in order to place the SVG image frame above the video. This technique is an upgrade from using the less flexible solution of absolute positioning.

The relevant styles for stacking the video and image with grid are as follows:

.holiday-card {
  display: grid;
  place-items: center;
  grid-template-areas: "card";
}

.holiday-card__video,
.holiday-card__img {
  grid-area: card;
}

.holiday-card__img {
  position: relative;
}

Through testing, I found relative positioning was necessary to ensure the image appeared above the video element.

While I won't go over every line in the styles, another key part is where we've set the .holiday-card wrapper to use aspect-ratio: 16/9. We're going to be requesting a video feed in HD format. The CSS property for aspect-ratio has seen increased support over just this past year, but we want to provide a fallback if it's not yet supported.

For that, we'll use a combination of the native CSS @supports feature and a custom property for setting the element's max-height. We'll update that value with JavaScript so that we can ensure it ends up with the 16/9 aspect ratio. We'll do that part when we complete our generator script.

@supports not (aspect-ratio: 16/9) {
  .holiday-card {
    max-height: var(--card-height, 30vh);
  }
}

Here's our styled card application with some other goodies like candy cane striping.

.demo {
  background-color: lightblue;
}

[class*="holiday-card"] {
  margin: 0;
}

.holiday-card {
  display: grid;
  place-items: center;
  grid-template-areas: "card";
  background-color: #222;
  aspect-ratio: 16/9;
  width: 100%;
  border-radius: 2rem 2rem 0 0;
  overflow: hidden;
}

@supports not (aspect-ratio: 16/9) {
  .holiday-card {
    max-height: var(--card-height, 30vh);
  }
}

.holiday-card__video,
.holiday-card__img {
  grid-area: card;
  display: block;
  max-width: 100%;
  max-height: 100%;
  /* required to prevent mobile portrait overflow */
  overflow: hidden;
}

.holiday-card__img {
  position: relative;
  width: 100%;
}

.holiday-card__actions {
  --color-green: #30a45e;
  --color-red: #e63a48;
  --color-light: #fcdce2;

  display: grid;
  grid-auto-flow: column;
  justify-content: center;
  gap: min(2rem, 4vw);
  padding: clamp(.5rem, 5%, 1rem);
  background: #e63a48;
  background-image: repeating-linear-gradient(45deg, var(--color-red) 0 2%, var(--color-light) 2% 5%);
  border-radius: 0 0 2rem 2rem;
  box-shadow: 0 0.5rem 0.5rem -0.25rem rgba(0, 0, 0, 0.35);
}

.holiday-card__actions button {
  all: unset;
  cursor: pointer;
  text-align: center;
  border-radius: 0.5em;
  border: 0.15em solid var(--color-light);
  color: #fff;
  background-color: var(--color-red);
  padding: 0.5em 0.75em;
  font-family: system-ui, sans-serif;
  font-size: 1.25rem;
  font-weight: bold;
  letter-spacing: 0.03em;
  box-shadow: 0.1em 0.15em 0.25em -0.08em rgba(0, 0, 0, 0.75);
}

.holiday-card__actions button:focus {
  outline: 2px dashed currentcolor;
  outline-offset: -7px;
}

.holiday-card__actions button:not(:disabled):hover {
  background-color: var(--color-green);
}

.holiday-card__actions button:disabled {
  cursor: not-allowed;
  opacity: 0.7;
  filter: grayscale(40%);
  box-shadow: inset 0.1em 0.15em 0.25em -0.08em rgba(0, 0, 0, 0.75);
}

#holiday-webcam-instructions {
  clip: rect(0 0 0 0);
  clip-path: inset(50%);
  height: 1px;
  overflow: hidden;
  position: absolute;
  white-space: nowrap;
  width: 1px;
}

Start Webcam Take Photo

The minimal script we first discussed is our starting point for the final holiday card generator.

With our card HTML now created, let's first create variables of all the pieces.

const card = document.querySelector(".holiday-card");
const cardImage = document.querySelector(".holiday-card__img");
const cardVideo = document.querySelector(".holiday-card__video");
const cardWebcamButton = document.querySelector(".holiday-card-webcam");
const cardPhotoButton = document.querySelector(".holiday-card-photo");

Recall we also setup a CSS custom property as a fallback for when CSS aspect-ratio wasn't supported, and it's time to supply that value. So now that we have the card element, we can compute the correct height to ensure the 16/9 ratio. This will only actually be set as the height if the browser doesn't support aspect-ratio due to how we set up the CSS using @supports.

card.style.setProperty("--card-height", card.offsetWidth * 0.5625 + "px");

An update to make is to improve the constraints to select for HD resolution and preference the front camera. The ideal key will request that the device provides the nearest available resolution if available. You can review the other available constraint parameters.

const constraints = {
  video: {
    width: { ideal: 1280 },
    height: { ideal: 720 },
    facingMode: "user"
  }
};

const handleSuccess = (stream) => {
  window.stream = stream;
  cardVideo.srcObject = stream;
};

const handleError = (error) => {
  console.error("Error: ", error);
};

const startWebcam = () => {
  navigator.mediaDevices
    .getUserMedia(constraints)
    .then(handleSuccess)
    .catch(handleError);
};

And now we'll add the click event listener to handle starting and stopping the webcam.

cardWebcamButton.addEventListener("click", (e) => {
  const webcamState = e.target.dataset.state;

  if (webcamState === "offline") {
    startWebcam();
  } else {
    stopWebcam();
  }
});

The stop function is yet to be created, but it relies on the fact we saved the stream into the window.stream object. The MediaStream API provides getVideoTracks() which returns an array of all found video streams. We're only expecting one, so we then call the stop() method. This immediately disconnects the video stream and the user's webcam will become inactive.

const stopWebcam = () => {
  if (window.stream) {
    window.stream.getVideoTracks()[0].stop();
  }
};

Now we have the basis of our functions but we haven't updated the state of our UI elements.

When the webcam is started successfully, we can do a few extra steps within handleSuccess() to make the "Take Photo" button available and also toggle the webcam button to "Stop" and its state to streaming.

const handleSuccess = (stream) => {
  cardWebcamButton.dataset.state = "streaming";
  cardWebcamButton.textContent = "Stop Webcam";
  cardPhotoButton.removeAttribute("disabled");
  cardImage.src = cardFrame;

  window.stream = stream;
  cardVideo.srcObject = stream;
};

One more piece slipped in, which was to reset the card img back to the SVG file path, which we need to add as a variable. We're doing this because when this function is called after the "Take Photo" action, the image source will be the saved screenshot. This is to reset to allow the video stream to be visible.

const cardFrame = "/12DaysCardFrame.svg";

We'll enhance stopWebcam() to handle the reverse state of the UI.

const stopWebcam = () => {
  cardWebcamButton.dataset.state = "offline";
  cardWebcamButton.textContent = "Start Webcam";
  cardPhotoButton.setAttribute("disabled", "disabled");

  if (window.stream) {
    window.stream.getVideoTracks()[0].stop();
  }
};

With our webcam start and stop working, it's time to enable taking the screenshot!

First, we need to create a canvas element. This can be added near our UI variables:

const cardCanvas = document.createElement("canvas");

Then, we'll start the function within the click event listener for the photo button. We prepare width and height variables that match the video resolution. This might seem like an odd choice, but it makes our final screenshot easier to take. There is a downside for a particular circumstance, which we'll discuss at the end.

cardPhotoButton.addEventListener("click", () => {
  const width = cardVideo.videoWidth;
  const height = cardVideo.videoHeight;

  // Setup Canvas
  cardCanvas.width = width;
  cardCanvas.height = height;
  const ctx = cardCanvas.getContext("2d");
});

Those dimensions are applied to the canvas element we created, and then comes the required step of obtaining the 2D context. The context is what we attach our drawing methods to for rendering the video and the frame SVG.

Our first item to render to canvas is the video, which is interestingly done via the canvas method of drawImage. The zeroes represent the x and y coordinates for placing the video on the canvas.

// Draw video
ctx.drawImage(cardVideo, 0, 0);

Following that, we begin to set up for rendering the SVG. We create a new canvas Image() and match the dimensions to the canvas. Then, set its source to the SVG file path.

// Create frame image
const frame = new Image(width, height);
frame.src = cardFrame;
frame.setAttribute("crossOrigin", "anonymous");

The crossOrigin attribute may not be necessary if your image is relative to your site, but might be needed if coming from a CDN. You can learn more about why it might be necessary for canvas images.

Creating the image in this way means it needs to load which requires time just like an image loaded into the DOM. Before we proceed with capturing the screenshot, we'll wait for the onload event of the image to ensure it's fully available. Otherwise only the video will be captured.

// Ensure frame loaded
frame.onload = () => {
  // Draw frame to canvas
  ctx.drawImage(frame, 0, 0, width, height);
  // Output final composed canvas to DOM img
  cardImage.src = cardCanvas.toDataURL("image/jpeg", 1.0);
};

The comments capture the steps, but after drawing the SVG we conclude the process by rendering the canvas contents as base64 encoded jpeg data into the card image src.

We selected jpeg as the file format so that it retains the video quality as much as possible and so that it is a type that the user can download and use as they wish. There are other options for toDataURL().

To close out the function, we need to trigger stopping the webcam. And since stopping the webcam disables the photo button, we'll move focus back to the "Start Webcam" button to prevent disorienting assistive technology users since disabled buttons are not focusable.

stopWebcam();
cardWebcamButton.focus();

Have fun trying out the generator!

const constraints = {
  video: {
    width: { ideal: 1280 },
    height: { ideal: 720 },
    facingMode: "user"
  }
};

const cardFrame = "/img/12DaysCardFrame.svg";
// Scoped to prevent working on the CSS demo :)
const card = document.querySelector(".scripted.holiday-card");
const cardImage = document.querySelector(".scripted .holiday-card__img");
const cardVideo = document.querySelector(".scripted .holiday-card__video");
const cardWebcamButton = document.querySelector(".scripted .holiday-card-webcam");
const cardPhotoButton = document.querySelector(".scripted .holiday-card-photo");
const cardCanvas = document.createElement("canvas");

card.style.setProperty("--card-height", card.offsetWidth * 0.5625 + "px");

const handleSuccess = (stream) => {
  cardWebcamButton.dataset.state = "streaming";
  cardWebcamButton.textContent = "Stop Webcam";
  cardImage.src = cardFrame;
  cardPhotoButton.removeAttribute("disabled");

  window.stream = stream;
  cardVideo.srcObject = stream;
};

const handleError = (error) => {
  console.error("Error: ", error);
};

const startWebcam = () => {
  navigator.mediaDevices
    .getUserMedia(constraints)
    .then(handleSuccess)
    .catch(handleError);
};

const stopWebcam = () => {
  cardWebcamButton.dataset.state = "offline";
  cardWebcamButton.textContent = "Start Webcam";
  cardPhotoButton.setAttribute("disabled", "disabled");

  if (window.stream) {
    window.stream.getVideoTracks()[0].stop();
  }
};

cardWebcamButton.addEventListener("click", (e) => {
  const webcamState = e.target.dataset.state;

  if (webcamState === "offline") {
    startWebcam();
  } else {
    stopWebcam();
  }
});

cardPhotoButton.addEventListener("click", () => {
  const width = cardVideo.videoWidth;
  const height = cardVideo.videoHeight;

  // Setup Canvas
  cardCanvas.width = width;
  cardCanvas.height = height;
  const ctx = cardCanvas.getContext("2d");

  // Draw video
  ctx.drawImage(cardVideo, 0, 0);

  // Create frame image
  const frame = new Image(width, height);
  frame.src = cardFrame;
  frame.setAttribute("crossOrigin", "anonymous");

  // Ensure frame loaded
  frame.onload = () => {
    // Draw frame to canvas
    ctx.drawImage(frame, 0, 0, width, height);
    // Output final composed canvas to DOM img
    cardImage.src = cardCanvas.toDataURL("image/jpeg", 1.0);
  };

  stopWebcam();
  cardWebcamButton.focus();
});

Webcam required to enable taking a photo for the holiday card Start Webcam Take Photo

Our generator is very likely to be functional on mobile given support of the APIs in use.

However, I noted that there's a potential issue with setting the canvas size to match the video size. And that is that on mobile in portrait mode, our video will not receive a stream using the 16/9 aspect ratio.

You may already be on a mobile device and have discovered how the camera stream in the preview has the portrait aspect ratio and is centered within the frame. This is thanks to CSS positioning the video element.

But, when you take a photo, a screenshot is successfully rendered but continues to have the portrait aspect ratio. This makes it look strange as the replaced still in our image element once again due to the CSS in place which causes it to falsely appear stretched. When you try to save it you'll see that the actual photo is portrait dimensions but this causes the SVG to stretch. This isn't completely terrible except that the demo SVG has text.

Consider this a challenge to you to work out how to resolve for mobile portrait mode! You could try to keep the portrait centered in the final image. Or, swap both the generator preview and the frame to accommodate a portrait orientation. Or whatever other solution makes sense to you!

Our holiday card generator is a solid base for building additional features for interacting with a user's webcam.

• Create preset classes that apply CSS filters to the video to pass into the canvas as well
• Provide a gallery of different frames to choose from, or different "stickers" or text phrases
• Enhance the webcam script to allow the user to choose the video source or change the resolution

If you expand this idea or create something new with what you've learned, tag #12DaysofWeb on Twitter or CodePen to let me know! I'd love to share your creations.

Links to the APIs are included where discussed in the tutorial, and I definitely recommend reviewing the full array of options they provide.

Two tutorials I found very useful in researching how to create this generator:

• Capture Audio and Video in HTML5 has an interesting history towards the APIs we used today and several examples of other things that can be done that are sure to inspire you
• Taking still photos with WebRTC on MDN covers similar things but in a more straightforward way

For advanced techniques, learn to create webcam effects from Adam Kuhn in this recording. Or go deep on learning real-time effects for images and video with a little extra help from WebGL.