Like a lot of engineers nowadays, I use Docker for my development environment. I have a Windows 11 machine with Windows Subsystem for Linux (WSL) and I use Docker Desktop for Windows. I also use Developer Containers with a volume for the source code to work with my blog, which means the files are not stored on the local file system but in a Docker volume. Occasionally, I need to access the files in the Docker volume for image creation. This post will show you how to access the files in the Docker volume on Windows.
First off, this post assumes you have Windows, either Windows 10 or Windows 11, with WSL and Docker Desktop installed. If you don’t have Docker Desktop installed, you can download it from Docker Hub.
Docker Volume Location
Update: 2025-01-11: Docker changed the locations of the volumes with release v26.1.4.
When you use Docker Desktop on Windows, the Docker volumes are stored in the WSL file system. The WSL file system is located at \\wsl$\ on the Windows file system.
If you want to see what version of docker you are running, you can open a command prompt or Powershell and run the following command.
Current Docker Version (v26.1.4 and Higher)
The Docker volumes are stored in the WSL file system at \\wsl$\docker-desktop\mnt\docker-desktop-disk\data\docker\volumes. You should see a folder for each volume you have created in Docker Desktop. Note: The wsl.localhost is no longer used in the path.
Previous Docker Version (v26.1.3 and Lower)
The Docker volumes are stored in the WSL file system at \\wsl.localhost\docker-desktop-data\data\docker\volumes. You should see a folder for each volume you have created in Docker Desktop.
Here is a screenshot of my Docker Desktop volumes.
Here is a screenshot of the WSL file system with the Docker volumes.
As you can see, there is a one to one mapping between the Docker Desktop volumes and the WSL file system. If you access these volumes regularly outside of the containers, you might want to rename the folders to something more meaningful.
Accessing the Volumes
If you do access the volumes a lot, you can create a symbolic link to the volumes folder. To do this, open a command prompt or Powershell as an administrator and run the following command.
Note: You can change the C:\Volumes to any folder you want to use.
Command Prompt
Version 26.1.4 and Higher
1
mklink /D C:\Volumes \\wsl$\docker-desktop\mnt\docker-desktop-disk\data\docker\volumes
Version 26.1.3 and Lower
1
mklink /D C:\Volumes \\wsl.localhost\docker-desktop-data\data\docker\volumes
PowerShell
Version 26.1.4 and Higher
1
New-Item -ItemType SymbolicLink -Path "c:\Volumes" -Target "\\wsl$\docker-desktop\mnt\docker-desktop-disk\data\docker\volumes"
Version 26.1.3 and Lower
1
New-Item -ItemType SymbolicLink -Path "c:\Volumes" -Target "\\wsl.localhost\docker-desktop-data\data\docker\volumes"
File Explorer
If you open up File Explorer, you should see a folder called Volumes in the root of the C:\ drive. This folder is a symbolic link to the Docker volumes.
Wrap Up
In this post, you learned where the Docker volumes are stored when using Docker Desktop on Windows. You also learned how to access the Docker volumes from the Windows file system and create shortcuts to them.
References
- Stack Overflow.
- Stack Overflow.
- Windows Subsystem for Linux (WSL).
Docker Desktop makes it simple to run Docker on your computer. One common task is to find where your data volumes are stored. Data volumes help keep your files safe even if a container is removed. In this guide, we will learn how to locate these data volumes in Docker Desktop. I use simple words and short sentences so it is easy to follow.
Introduction
Docker volumes store your data outside of the container. They let you keep information safe between container restarts. When you work with Docker Desktop, you may wonder where these volumes live. Unlike files on your normal computer, volumes are hidden inside Docker’s internal storage. This article shows you several ways to find them.
Understanding how Docker volumes work is very helpful. You can read more about this in our article on what are docker volumes and how do they work? This gives you a good background.
Docker Desktop and Its Storage
Docker Desktop runs Docker in a special virtual machine. On Windows and macOS, this virtual machine hides many file system details. Data volumes are kept inside this VM and are not directly visible in your normal file explorer.
Because of this, the easiest way to locate volumes is by using Docker commands. The Docker CLI lets you list, inspect, and manage volumes. If you want a more complete guide on creating and using volumes, see how to create and use docker volumes.
Listing Your Data Volumes
The first step is to list all your volumes using the Docker CLI. Open a terminal and run:
docker volume ls
Enter fullscreen mode
Exit fullscreen mode
This command prints a list of volumes with their names. Each volume has a unique name, which you can use later to inspect it in more detail.
For example, you might see output like this:
DRIVER VOLUME NAME
local pgdata
local myapp_volume
Enter fullscreen mode
Exit fullscreen mode
This shows that there are volumes named pgdata and myapp_volume. To learn more about each volume, you can inspect them.
Inspecting a Specific Volume
Once you have a volume name, use the following command to get details:
docker volume inspect <volume_name>
Enter fullscreen mode
Exit fullscreen mode
Replace <volume_name> with the name from your list. For example:
docker volume inspect pgdata
Enter fullscreen mode
Exit fullscreen mode
This command returns a JSON object with information about the volume. It shows the driver, the mount point inside the Docker VM, and other details.
By reading the output, you can see the location where Docker Desktop stores the volume data. The mount point is a path on the VM’s internal file system. Although you cannot easily browse this path from your host machine, you now know where the data lives inside Docker.
For more on listing and inspecting volumes, check out our guide on how to list and inspect docker volumes.
Accessing Volume Data
In many cases, you do not need to access the data files directly. Instead, you can use containers to work with your data. However, if you need to open a shell inside a container and check the files, you can run a temporary container that mounts the volume.
For example, to open a shell and explore a volume named myapp_volume, run:
docker run --rm -it -v myapp_volume:/data alpine sh
Enter fullscreen mode
Exit fullscreen mode
This command does the following:
- —rm removes the container when you exit.
- -it opens an interactive terminal.
- -v myapp_volume:/data mounts the volume to the container’s /data folder.
- alpine sh starts a simple Alpine Linux shell.
Once inside, you can list files in the /data folder to see your volume’s contents. When you are done, type exit to leave the shell.
For advanced users who wish to mount directories from the host to containers, you may also find the article on how to mount host directories to docker containers useful. It explains the difference between volumes and bind mounts.
Using Docker Desktop UI
Docker Desktop also provides a graphical user interface (GUI) to manage containers and images. While the GUI does not show volume file paths directly, it can help you see which volumes are in use.
- Open Docker Desktop.
- Go to the Volumes tab in the Dashboard.
- Here, you can see a list of volumes. Click on one to view its details.
This method is useful if you prefer to use a point-and-click approach rather than the command line. It also gives you a quick overview of all storage used by Docker.
If you are not sure how to use Docker Desktop, check out our article on what is Docker Desktop and how do you use it? for a basic introduction.
Where Are Volumes Stored on Your Host?
Because Docker Desktop runs inside a VM, the actual files are stored in a hidden area. On macOS, the files are usually in a virtual disk image managed by Docker. On Windows, they are inside a Hyper-V or WSL 2 distribution.
For most users, it is best not to change these settings manually. Instead, work with volumes using Docker commands or the Docker Desktop UI. The system handles the details for you. Knowing that the data is safe inside the VM is usually enough.
Why Locating Volumes Matters
There are several reasons why you might need to locate your Docker volumes:
- Troubleshooting: If you have problems with a container, inspecting the volume can help diagnose the issue.
- Backups: You may want to back up important data stored in a volume.
- Data Migration: When moving data from one system to another, knowing the volume details is helpful.
Understanding where your data lives and how it is managed is a key part of using Docker well. For more insight into docker volumes, you might also review our guide on what are docker volumes and how do they work?
Best Practices
Here are some best practices for working with Docker volumes in Docker Desktop:
-
Use Named Volumes:
Always use named volumes instead of anonymous ones. This makes it easier to track and manage your data.
-
Regularly Inspect Volumes:
Use
docker volume lsanddocker volume inspectto monitor your volumes. This helps you understand their size and usage. -
Backup Important Data:
Create regular backups of critical volumes. Even if the files are hidden inside the VM, you can export the data using a temporary container.
-
Use the Docker Desktop UI:
The Dashboard gives a quick view of your volumes. It is a helpful tool for new users who are not comfortable with the CLI.
For more details on how to work with volumes and keep your data safe, you may refer to our article on how to create and use docker volumes.
Conclusion
Locating data volumes in Docker Desktop is a key skill for anyone working with Docker. By using simple commands like docker volume ls and docker volume inspect, you can easily list and find your volumes. The Docker Desktop UI also offers a simple way to view volume information. Although the actual files are stored inside a hidden VM, you can work with them safely using Docker commands.
To learn more about Docker storage, check out our guides on:
- what are docker volumes and how do they work?
- how to create and use docker volumes
- how to list and inspect docker volumes
- what is Docker Desktop and how do you use it?
- how to mount host directories to docker containers
By following these simple steps and best practices, you can confidently manage your Docker volumes. Keep practicing, and over time these tasks will become second nature. Happy containerizing and good luck with your Docker projects!
“When running Windows Subsystem for Linux (WSL) using Docker Desktop, the Docker volumes are typically located in a specific directory within its Linux file system, providing an advantageous solution for persistent data storage.”To generate a summary view of where Docker volumes are located when running Windows Subsystem for Linux (WSL) using Docker Desktop, we first need to understand the concept of how Docker interacts with WSL. The docker volumes are actually stored on your local machine’s hard drive and not in the WSL file system.
The Docker Desktop for Windows uses Linux containers on Windows (LCOW) and places it’s Docker volumes in a directory path, generally found at
C:\Users\[Your User Name]\AppData\Local\Docker\wsl\data-ext4.vhdx
or
%APPDATA%\Docker\wsl\data-ext4.vhdx
. Note that this is a VHDX file, which stands for Virtual Hard Disk format that is used by Windows for virtual machines. This VHDX file is where the Docker WSL data resides.
Here’s a table representing the generated summary:
| Element | Description |
|---|---|
| Data Location in Docker Desktop for Windows | C:\Users\[Your User Name]\AppData\Local\Docker |
| VHDX File Name | wsl\data-ext4.vhdx |
| What’s inside the VHDX? | The Docker WSL data resides here |
The aforementioned information shows that locating Docker volumes while using Docker Desktop with WSL is fundamentally tied up with understanding the way Docker establishes its communication and storage within the hosting environment. The Docker volumes’ actual location lies outside the Linux VM created by WSL, instead, they are part of your Windows file system. Therefore, interacting with Docker volumes within WSL involves navigating through the path mappings betweenWindows and WSL.
Thus, when you’re running WSL with Docker Desktop, you’re essentially accessing Docker volumes indirectly – Docker volume data exists on your Windows hard drive while being accessed and manipulated within your Linux VM. It’s a complex integration but gives us the versatility of working in a Linux environment with the underlying support of a Windows system.
For further reading, I would highly recommend looking into the Docker and WSL documentation provided on their official websites, both of which offer comprehensive coverage on these topics:
- Docker WSL Documentation
- Microsoft WSL Documentation
Understanding Docker volumes and their location, particularly when using WSL (Windows Subsystem for Linux) with Docker Desktop, can initially come across as challenging. But do not worry. Let’s break it down!
Docker volumes are the go-to method for persistent data provision and sharing between containers in Docker, bypassing the ephemeral nature of containers. Docker volumes exist outside traditional union filesystem and have several advantages:
- They are efficient both for (I/O) operations and storage.
- Data persists beyond the lifecycle of individual containers.
- Information stored on volumes is easily shared and accessed by multiple containers.
On Docker Desktop for Windows installations, the location of your Docker volumes can differ depending on whether you’re using a Linux or Windows container. For instance, if you create a volume without specifying a source, it’ll be added to the default directory:
<docker installation path>/volumes/<volume name>
To locate Docker volumes when running WSL2 (Windows Subsystem for Linux 2) using Docker Desktop, you first need to understand that Docker Desktop stores Linux-related files inside a utility VM (Virtual Machine). By default, you cannot access this VM via more traditional methods like you could other VMs.
However, Docker volumes can still be located through WSL2 using your preferred terminal or command line interface. You may navigate to the specified Docker volume storing system using a command similar to:
cd /mnt/wsl/docker-desktop-data/version-pack-data/community/docker/volumes
Remember, the path might change based on Docker’s internal organization or configurations.
With an understanding of Docker Volumes’ benefits and how to locate them when utilizing Docker Desktop close at hand, you’re now equipped to possibly harness and leverage these data structures in a way that complements your container-based workflow. Gaining such knowledge is crucial as a professional developer, and when you look under the hood a little, complexities surrounding Docker volumes decrease considerably.
It’s always recommended for coders to delve into official Docker documentation whenever unsure about implementation specifics or best practices. The person behind the code can make all the difference!
The Windows Subsystem for Linux (WSL) and Docker Desktop have a critical connection when it comes to running distributed applications on a Windows workstation. To fully understand where Docker Volumes are located in this setup, we first need to invest some time exploring the symbiotic relationship between WSL and Docker Desktop.
A Peek into WSL
WSL is an ingenious compatibility layer designed to enable the execution of Linux binary executables natively within Windows 10.
wsl --set-version Ubuntu-20.04 2
It effectively runs a GNU/Linux environment — complete with command-line interface, system utilities, and applications — directly on Windows, unmodified, without the overhead associated with using a traditional virtual machine or dual boot setup.
Docker Desktop’s Marriage with WSL
Docker Desktop for Windows is a Docker Community Edition (CE) application created by Docker Inc. This version of Docker Desktop uses WSL 2 (based on the Linux 4.19 kernel) to run its docker daemons instead of Moby VM which was used in the previous versions of Docker Desktop utilizing Hyper-V.
docker run -d -p 80:80 docker/getting-started
By integrating Docker Desktop with WSL, developers can benefit from a consistent, cross-platform experience. With Docker Desktop permissions tied specifically to your WSL installation, Docker containers act as if they were native apps on your work station.
Where oh Where Are My Docker Volumes?
When you properly integrate Docker with WSL, volume mounts work out of the box. In fact, any directory path that works with any command in WSL will serve equally well as a volume mount for docker.
docker volume create my-vol echo ' <my> "html file content here." </my> ' > /mnt/c/Users/hello.html docker run -d --name devtest -v my-vol:/app nginx:latest
In the above example, hello.html will be in ‘my-vol’ volume in ‘/app’ folder. Now, where exactly is this Docker Volume?
Generally, all Docker data, including volumes, gets stored within /var/lib/docker/ or equivalent directory path based on the OS you’re using. However, due to WSL-Docker integration, WSL controls the Linux namespace for Docker. Which means, even though Docker sees its volume inside /var/lib/docker/volumes/, from the host perspective, this corresponds to the Docker Desktop virtual hard disk storage location.source
This storage can be accessed in Windows through File Explorer -> Network -> wsl$. Provided your WSL & Docker Desktop configurations are correctly set up, your Docker bound volumes should rest within these directories. Keep in mind Docker stores only non-empty named and anonymous volumes while bind mounts may be flat files or even non-existent paths.
Understanding the delicate interplay between WSL and Docker Desktop helps us manage our workflow better and perform required operations, like accessing the Docker volumes, more efficiently.Docker volumes, when running in a WSL (Windows Subsystem for Linux) environment, are particularly intriguing as they make use of Docker’s inbuilt volume management system. These volumes provide a more flexible and effective medium for data persistence compared to bind mounts, which allows for efficient data access even if the container stops.
The Location of Docker Volumes in WSL
Locating Docker volumes in WSL can be a bit intricate because they aren’t placed in your regular Windows or WSL directories. Rather, they are situated in a virtual hard disk (ext4.vhdx) on Docker Desktop’s virtual machine. The following path typically represents the location:
\\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes
When you navigate through these paths, you’ll find various folders named ambiguously according to their unique identifiers. Each folder aligns with each individual Docker volume that you have created. Inside these folders, you will locate your actual persistent data associated with your Docker containers.
However, bear in mind that interacting with files in this location from your Windows filesystem could potentially lead to issues with file metadata and permissions.
Working With Docker Volumes in WSL
Using the Docker CLI, you can create, list, delete, and manage the Docker volumes from the WSL terminal.
To create a new volume, use the following command:
docker volume create my_volume
Listing all existing volumes is done through this command:
For removing a Docker volume:
docker volume rm my_volume
When running a Docker container in WSL while specifying a volume, the syntax should correspond to the following format:
docker run -v my_volume:/path/in/container -it Ubuntu bash
In this format, ‘my_volume’ signifies your Docker volume name and ‘/path/in/container’ refers to the path where this volume would mount inside the container.
A Useful Tip: Using Bind Mounts for File Access
For scenarios wherein direct and immediate file interaction is required between your Windows OS and Docker Containers, another feature known as bind mounts can be utilized. Here’s how you can do it while running a container:
docker run -v /c/Users/username/Documents/my_directory:/path/in/container -it Ubuntu bash
The ‘-v’ flag specifies the source directory on your host system and the target directory inside your container where the mount point should reside.
It’s crucial to note that Docker interprets Windows paths very literally. Therefore, the Windows drive has to be declared (/c/) instead of declaring C:\.
To gain a more thorough understanding of Docker volume intricacies while working within WSL environments, the official Docker documentation(source) comes in handy.That’s a great question! Understanding the location of Docker Volumes when using Docker Desktop on Windows Subsystem for Linux (WSL) is crucial for managing your data efficiently.
Like UNIX-based systems, Docker stores volumes within the
directory. This is easy to find on a native UNIX system. However, in our case, as we’re using WSL with Docker Desktop on a Windows machine, things require a little more digging.
When running Docker Desktop, volumes are nestled within the Docker Desktop VM. The actual path is a hidden system folder in Windows, and most tools won’t show it by default.
We can access this information by typing the following command in the terminal:
shell
wsl -l -v
And you may see something like this:
shell
NAME STATE VERSION
* docker-desktop Running 2
docker-desktop-data Running 2
The
is where the Docker Desktop VM stores volume data.
To browse this file system, we can use another command to list all files under
:
shell
wsl -d docker-desktop ls /var/lib/docker/volumes
With extensive digging, you’ll see that Docker volumes’ real location is somewhere deep in Windows’ system folders, along these lines:
C:\Users\YourUserName\AppData\Local\Docker\wsl\data\ext4.vhdx
. But without specific drag-and-drop capability between the Windows host system and the Docker guest environment or capacity for translating the VHDX filesystem, accessing this directly isn’t recommended or even practical.
Such an indirect location has functional benefits. It protects your system integrity and security, ensuring buffered interactions between your containers and the raw power of your base OS.
If you need to probe more, consider mounting your desired Windows directories into the Docker containers. This strategy allows guests (containers) to read/write, creating a convenient interaction channel without jeopardizing safety, and also making managing volumes directly a lot easier.
To mount, use a command like:
docker run -v c:/Users/path-to-directory-on-windows:/app imageName
This instructs Docker to map a local filesystem mount to a path in the container (/app). Any changes made to this destination will be written back to the source directory.
Understanding how Docker handles volumes and where it actually locates them (in the context of WSL on Docker Desktop) helps us harmoniously manage both host and virtualized environments, improving all-around robustness and maintainability.When working in a WSL environment using Docker Desktop, the navigation of Docker volume locations can seem confusing. But once you get your bearings right, it becomes quite smooth and intuitive.
All the significant directories, such as volumes, are stored under a default docker-desktop-data VM instance on WSL controlled by Docker Desktop. The data is primarily located in `/var/lib/docker/`, but it’s not accessible from the WSL Linux distros directly due to the design intention to encapsulate the data and prevent accidental modification.
To navigate the Docker volumes:
Step 1: Access Docker Desktop Data Instance
Open a new WSL terminal and switch to the Docker Desktop Data instance with this command:
wsl -d docker-desktop-data
Step 2: Navigate to the Docker Directory
Once you are in the Docker Desktop instance, go to the Docker directory:
This specific location allows us to explore all sorts of Docker configurations, including volumes.
Step 3: Open Volumes Directory
Navigate to the volumes directory:
Within this directory, each folder correlates to a different Docker volume that exists within your system, named after the unique identifier that Docker assigns to each volume upon creation.
Important Note:
In case you have made configuration modifications for Docker-Desktop, like setting custom paths or changing the location for the Docker Desktop resources, the path will mirror these changes. More information on how the directory structure looks inside `/var/lib/docker` can be found on Docker’s official documentation.
A Word on Container Storage:
It’s also key to remember when using Docker with WSL that containers and images are stored separately under Docker virtual machines (docker-desktop and docker-desktop-data) rather than the specific WSL distribution.
Cyberdrk in their GitHub post, “Running Docker on WSL”, provides an excellent overview of where Docker stores container image files when running inside a WSL environment.
With a good understanding of these locations, you can maximise your productivity and efficiency when dealing with Docker volumes in a WSL environment!One of the main pillars of Docker technology is building amicable ecosystems for developers by taking two fundamental insights – volumes, which store data generated by and used by docker containers and images, which are lightweight and standalone executable packages. In Docker, these fundamentally contribute to containerization utility.
With Windows Subsystem for Linux (WSL) operating within Docker Desktop, here’s how they interplay:
Docker Images And Containers:
Docker employs a Union File System that allows layers of files to overlay. For instance,
This docker image has multiple read-only layers and each directive creates a new layer.
The real magic happens when you initialize the Docker container from an image. It adds a thin, modifiable layer atop the Image’s read-only layers which makes the containers incredibly lightweight.
Docker Volumes and Docker Containers:
When you create a Docker container, any data that the container generates will vanish when the container is deleted unless you use Docker volumes. These are preferred mechanisms that persist data generated and used by Docker containers.
So essentially, Docker volumes can be shared among multiple containers and their content exists as long as the volume exists.
To illustrate this, we can run a basic alpine based Docker container with a volume named ‘vol’.
docker run -it --name=alpine-test -v vol:/root/data alpine sh
Now, any data inside ‘/root/data’ in the container will remain even if the container is deleted, thanks to ‘vol’. This makes Docker volumes critical for data persistence.
Understanding Docker Device Mapper in WSL:
In a Docker on Linux setup, the Docker daemon directly interacts with the system. However, when using Docker Desktop on Windows, it uses a virtual machine (Hyper-V or WSL2 based) to build and run Docker containers, meaning Docker isn’t interfacing directly with Windows File System but rather through a Linux VM.
When operating within WSL using Docker Desktop, the space that Docker reserves for images, containers, volumes etc., is managed by Docker’s device mapper storage driver.
As shown above, the device mapper divides the physical storage into multiple logical blocks called ‘Data Blocks’. Each block corresponds to a single layer of a Docker image or container. Volume data gets propagated over these blocks as well.
This ‘Data-Block Pool’, where your Docker bits reside in, can be controlled using the preferences of Docker Desktop.
Locating Docker Volumes in WSL:
However, finding whether your ‘Data Block Pool’ lies isn’t really straightforward in Windows. If you’re using the new WSL2 backend (available in Docker Desktop 2.x), your volumes are stored in the distro’s ext4 vhd file at:
\\wsl$\«distro»\var\lib\docker\volumes
Where «distro» would be your running distro like ‘Ubuntu-20.04’.
The ‘\\wsl$\’ approach allows accessing your WSL Linux files in Windows explorer which makes management easier.
Ultimately, understanding the interlink between volumes, containers, and images and where they are located in Docker, particularly when using WSL, is essential to make the most out of Docker’s powerful containerization utility.When running WSL (Windows Subsystem for Linux) with Docker Desktop, the concept of data persistence is definitely something worth understanding to ensure that your applications and services continue to function as expected even after you close, reopen, or modify containers—or even reboot your system completely. One way this data persistence is achieved in Docker is by using “volumes”, which can be shared and reused among different containers.
So, where are these Docker volumes located when we run Docker on a WSL environment? They actually reside within the virtualized hard disk controlled by Docker Desktop. This location isn’t typically designated within the WSL filesystem but rather inside Docker’s own managed area.
| Type | Location |
|---|---|
| Docker volume | /var/lib/docker/volumes/… |
This path ‘/var/lib/docker/volumes/’ is where the majority of Docker volumes can be found when running Docker Desktop on a WSL environment. Each directory in this path corresponds to a particular volume mounted by a container. Although it appears to be a standard Unix filesystem path, remember that behind the scenes, the WSL and Docker Desktop are managing this area themselves, so consider it a specialized environment.
For retrieval and interaction with the data of Docker volumes on a larger scale, Docker’s CLI (Command Line Interface) provides commands such as
To list all currently registered Docker volumes
And
docker volume inspect [volume_name]
This command gives detailed information about a specific volume including its location.
While working with Docker Desktop and WSL, throughout time, you’d possibly gain large volumes containing varying types of application-related data. It becomes necessary then to manage and monitor these volumes effectively. A wealth of open-source tools like Portainer and monitoring solutions such as Datadog exist allowing you to interact with and visualize your Docker volumes. And whilst running WSL, don’t forget the power of simple, built-in Linux commands such as ‘du -sh’ to check the usage size of a folder or volume at any time.
In short, Docker volumes through Docker desktop with WSL give developers a robust and consistent method for handling data persistence across various scenarios, irrespective of the container state. The combination makes a compelling case for developing and running Docker-infused applications right from your Windows workstation.
References:
1. Docker Documentation: Use volumes
2. Microsoft: About the Windows Subsystem for LinuxThe Docker’s data volumes are a crucial part of Docker infrastructure, offering persistent and sharable storage model for Docker containers. It’s essential to recognize where these volumes exist so you can manage them effectively.
When using Docker on Windows through Docker Desktop, the volume information is all stored within the Virtual Machine that Docker Desktop provisions. However, it is imperative to note that this VM exists in a hyper-abstracted state and doesn’t mount directly onto your Windows filesystem.
See also
That being said, most modern Docker Desktop implementations lean towards WSL 2-based backend. Here’s a simple way to find Docker Volumes when running WSL using Docker Desktop:
The location of Docker volumes for Docker Desktop in a Windows environment with WSL 2 usually resides under the Linux file system at:
You can navigate to this folder using the command-line (CLI) in Windows Subsystem for Linux (WSL). Simply open your favorite Linux distribution (Ubuntu, Debian, etc.) on WSL and enter the following command:
cd /var/lib/docker/volumes
There, you will have access to directories correlating to each existing Docker volume. You may require root user permissions to access or modify the files under these directories, however.
It’s important to remember that managing Docker volumes often involves interactions with the Docker engine, typically via Docker CLI commands. For instance:
* Listing all volumes:
* Creating a volume:
$ docker volume create my-vol
* Examining a volume:
$ docker volume inspect my-vol
* Removing a volume:
$ docker volume rm my-vol
Though accessing the file system directly gives an operational level control, interfacing via the Docker engine commands helps ensure a greater consistency and less chance for unwanted side-effects.
I mustn’t ignore to mention the excellent documentation of Docker volumes available at Docker’s official website, where you can find detail insights about volume management in Docker environments.
Remember, Docker Desktop on Windows with its partnership with the Windows Subsystem for Linux simplifies much of this process, providing a more native-feeling Docker experience for Windows. Working together, they strive to maintain Docker’s principle of “Build, Ship, and Run Any App, Anywhere.”<
The impact of mount points in locating your Docker volume is significant, especially when running Windows Subsystem for Linux (WSL), using Docker Desktop. In a Docker file system, a mount point is where the data volume is mounted in the container. Mount points can significantly affect where and how you find your Docker volumes on WSL using Docker Desktop.
To brief it up, Docker offers persistent storage through the use of volumes. Docker volumes hold writable disk areas that are separate from the default Union File System. They enable data to persist even after a container stops. But finding these volumes becomes slightly complex.
Typical Location of Docker Volumes
In a typical non-WSL setup, Docker volumes are often located at
. For example, if you create a volume named myvolume, its data will be stored in
/var/lib/docker/volumes/myvolume/_data
.
# example docker volume create myvolume docker run -d -v myvolume:/container/path --name myContainer myImage
Locating Docker Volumes with WSL Using Docker Desktop
However, once we introduce WSL into the equation, locating Docker volumes changes entirely. Since Docker Desktop uses a virtual machine behind the scenes, your Docker volumes are not found in the traditional spot within the WSL filesystem.
Instead, your Docker volumes are stored inside the Docker Desktop Linux VM which Windows doesn’t have direct access to. If you’re looking for your volumes within WSL, this could lead to confusion and difficult debugging processes. Thus, it’s important to remember this change when running WSL with Docker Desktop.
Accessing Your Docker Volumes
Although your volumes might seem out of reach due to this structure change, there’s a way to access them without needing to mess around with the underlying VM. You need to use the
command to copy files between the Docker container and the host file system.
# Copy a file from docker container to host docker cp <containerId>:/file/path/within/container /host/path/to/copy/file/to # Copy a file from host to docker container docker cp /host/path/to/copy/file/from <containerId>:/file/path/within/container
While slightly more convoluted than just accessing a directory in your filesystem, this method provides a secure and reliable way of interacting with your Docker volumes within the WSL environment.
So, the mount points do indeed have an impact on locating your Docker volumes, especially when it comes to running WSL with Docker Desktop.
They make accessing the volumes directly seemingly impossible. Understanding how to navigate it saves time and prevents inevitable frustrations. Remember to use the docker commands wisely to interact with your data.
You may read further on the Docker documentation Docker WSL integration.
When you’re operating in a Windows environment and using the Windows Subsystem for Linux (WSL), working with Docker Volumes can be somewhat different compared to a traditional Linux environment. There are several points that need to be noted.
Docker Volumes Location
First and foremost, the location of Docker volumes might feel a little bit strange if you are used to Linux systems. In Windows environments, Docker Desktop stores Linux files inside a virtual hard disk (VHD). This VHD is labeled as “ext4.vhdx” and it’s typically found within your user directory under:
Docker\wsl\data\ext4.vhdx
. If you navigate into this file from your Windows Explorer, however, you will not find anything recognizable because it’s a specific type of Virtual Hard Drive containing an ext4 file system, which is a standard file system for most Linux distributions.
| Operating System | Docker Volumes Path |
|---|---|
| Windows 10 (With WSL) | C:\users\your_username\Docker\wsl\data\ext4.vhdx |
In order to view or manipulate files within these Docker volumes, you have to go through Docker Desktop’s WSL interface. On your terminal, you can do so by typing in:
command. Now, your terminal should be within the Docker instance of WSL. The function of Docker volumes still remains the same, meaning they store data generated by and used by Docker containers.
wsl -d docker-desktop cd /var/lib/docker/volumes/ ls
The
path in your shell is where you’ll find all the Docker volumes for your containers. Every Docker volume is identified by a long alphanumeric string, which corresponds to a folder within this directory.
Volume Operations
Other Docker operations such as creating, deleting, and managing Docker volumes fall under the same Docker CLI commands whether on the console, terminal, or CMD.
Create a Docker volume.
docker volume create my_volume
List Docker volumes.
Delete a Docker volume.
docker volume rm my_volume
Moving Data Between Windows and Docker
Sometimes, you may want to share files between your Windows environment and Docker containers. To accomplish this, you are advised to use Docker bind mounts. Bind mounts will have access to the file system path you point towards. Hence, you can easily move data around your local machine and Docker containers seamlessly. Do note that bind mounts rely heavily on your file system’s pathing. Here’s an example of how a Docker container can start and have a shared directory with your local system:
docker run -d --name=my_container -v C:/Users/Your_user/Documents:/shared_folder my_image
This command mounts the host directory,
C:/Users/Your_user/Documents
, to the container under the path
.
Synchronization
Relying on Docker Desktop’s WSL 2 backend will greatly enhance your Docker experience on Windows. Since it works towards synchronization between Linux and Windows, Docker changes made within WSL will reflect on your Docker Desktop dashboard immediately and vice versa.
Your Responsibility
As with any tool, the main pointers revolve around understanding core functionalities and best practices. Read up on and keep close tabs on the versions and capabilities of both Docker and WSL updates. Microsoft and Docker are continually enhancing these tools and new versions may substantially streamline your Docker volume management and overall containerization tasks on Windows.
References:
1. Docker Documentation: https://docs.docker.com/
2. Microsoft WSL Documentation: https://docs.microsoft.com/en-us/windows/wsl/
Analyzing and understanding the location of Docker volumes when running WSL using Docker Desktop is an essential aspect for any developer working with such configuration. Docker Desktop for Windows uses WSL 2 to run its containers, thus resulting in storing its data as part of the Linux filesystem.
Typically, your Docker volumes, which serve as the designated locales for data persistency, reside within the
the directory inside your Docker virtual machine. This data storage mechanism assures you can consistently access and manage your data across various container life cycles.
| Docker Volume Location |
|---|
/var/lib/docker/volumes/ |
However, given Docker’s compatibility with the WSL filesystem, you might wonder how this impacts the storage positioning in the mixed environment. When it comes to WSL2, the Docker filesystem symmetry is maintained. Consequently, the Docker Desktop volumes when running on Windows Subsystem for Linux (WSL) remain at the same location, within the Linux system, stored under the
.
But remember that these directories are not directly accessible from the Windows host filesystem. So, to access or navigate these files from Windows, utilize a Linux command-line interface, such as the built-in one in WSL, or you’ll also find Docker volume inspection commands rather useful.
Having adequate knowledge about Docker’s structures, like volumes, is crucial because they form the nuts and bolts of containerization. As a coder, whether you are developing, testing, or deploying applications using Docker and WSL, this comprehension definitely improves your efficiency while handling data persistence and storage structures.
It’s also worth noting that Docker continues to evolve to offer more seamless integration to its users among diverse environments. Keeping track of Docker’s updates, especially those relating to these key aspects, helps you adapt to potential changes quickly.
To dive deeper into understanding docker volumes management, consider reading the official Docker documentation [link]. It will allow you to uncover more detailed insights about managing docker volumes, including exploring different docker volume drivers, and managing volumes through docker CLI commands effectively.
Intro
“Where are the Docker volumes?”
This question comes up a lot on the Docker forum.
There is no problem with curiosity, but this is usually asked when someone wants to
edit or at least read files directly on the volume from a terminal or an IDE,
but not through a container. So I must start with a statement:
Important
You should never handle files on a volume directly without entering a container,
unless there is an emergency, and even then, only at your own risk.
Why I am saying it, you will understand if you read the next sections.
Although the original goal with this tutorial was to explain where the volumes are,
it is hard to talk about it without understanding what the volumes are
and what different options you have when using volumes.
As a result of that, by reading this tutorial, you can learn basically
everything about the local volumes, but you can also
search for volume plugins on Docker Hub.
Where does Docker store data?
Before we talk about the location of volumes, we first have to talk about
the location of all data that Docker handles.
When I say “Docker”, I usually mean “Docker CE”.
Docker CE is the community edition of Docker and can run
directly on Linux. It has a data root directory, which is the following by default:
You can change it in the daemon configuration, so if it is changed on your system,
you will need to replace this folder in the examples I show.
To find out what the data root is, run the following command:
docker info --format '{{ .DockerRootDir }}'
In case of Docker Desktop of course, you will always have a virtual machine,
so the path you get from the above command will be in the virtual machine.
What is a Docker volume?
For historical reasons, the concept of volumes can be confusing.
There is a page in the documentation
which describes what volumes are, but when you see a Compose file or a docker run command,
you see two types of volumes, but only one of them is actually a volume.
Example Compose file:
services: server: image: httpd:2.4 volumes: - ./docroot:/usr/local/apache2/htdocs
Did I just define a volume?
No. It is a bind mount.
Let’s just use the long syntax:
services: server: image: httpd:2.4 volumes: - type: bind source: ./docroot target: /usr/local/apache2/htdocs
The “volumes” section should have been “storage” or “mounts” to be more clear.
In fact, the “docker run” command supports the --mount option in addition to
-v and --volume, and only --mount supports the type parameter
to directly choose between volume and bind mount.
Then what do we call a volume? Let’s start with answering another question.
What do we not call a volume? A file can never be a volume. A volume is always a
directory, and it is a directory which is created by Docker and handled by Docker
throughout the entire lifetime of the volume. The main purpose of a volume is
to populate it with the content of the directory to which you mount it
in the container. That’s not the case with bind mounts. Bind mounts just
completely override the content of the mount point in the container, but at least
you can choose where you want to mount it from.
You should also know that you can disable copying data from the container to your
volume and use it as a simple bind mount, except that Docker creates it in the Docker
data root, and when you delete the volume after you wrote something on it, you will lose
the data.
volumes: docroot: services: server: image: httpd:2.4 volumes: - type: volume source: docroot target: /usr/local/apache2/htdocs volume: nocopy: true
You can find this and other parameters in the
documentation of volumes in a compose file.
Scroll down to the “Long syntax” to read about “nocopy”.
Custom volume path
Custom volume path overview
There is indeed a special kind of volume which seems to mix bind mounts and volumes.
The following example will assume you are using Docker CE on Linux.
volume_name="test-volume" source="$PWD/$volume_name" mkdir -p "$volume_name" docker volume create "$volume_name" \ --driver "local" \ --opt "type=none" \ --opt "device=$source" \ --opt "o=bind"
Okay, so you created a volume and you also specified where the source directory is (device),
and you specified that it is a bind mount.
Don’t worry, you find it confusing because it is confusing.
o=bind doesn’t mean that you will bind mount a directory into the container,
which will always happen,
but that you will bind mount the directory to the path where Docker would have
created the volume if you didn’t define the source.
This is basically the same what you would do on Linux with the mount command:
mount -o bind source/ target/
Without -o bind the first argument must be a block device.
This is why we use the “device” parameter, even though we mount a folder.
This is one way to know where the Docker volume is.
Note
Even the the above example assumed Linux, custom volume path
would work on other operating systems as well, since Docker
Desktop would mount the required path into the virtual machine.
Let’s just test if it works and inspect the volume:
docker volume inspect test-volume
You will get a json like this:
[ { "CreatedAt": "2024-01-05T00:55:15Z", "Driver": "local", "Labels": {}, "Mountpoint": "/var/lib/docker/volumes/test-volume/_data", "Name": "test-volume", "Options": { "device": "/home/ta/test-volume", "o": "bind", "type": "none" }, "Scope": "local" } ]
The “Mountpoint” field in the json is not the path in a container, but the path where
the specified device should be mounted at. In our case, the device is actually a directory.
So let’s see the content of the mount point:
sudo ls -la $(docker volume inspect test-volume --format '{{ .Mountpoint }}')
You can also check the content of the source directory:
Of course, both are empty as we have no container yet.
How would Docker know what the content should be?
As we already learned it, we need to mount the volume into a container
to populate the volume.
docker run \ -d --name test-container \ -v test-volume:/usr/local/apache2/htdocs \ httpd:2.4
Check the content in the container:
docker exec test-container ls -lai /usr/local/apache2/htdocs/
Output:
total 16 256115 drwxr-xr-x 2 root root 4096 Jan 5 00:33 . 5112515 drwxr-xr-x 1 www-data www-data 4096 Apr 12 2023 .. 256139 -rw-r--r-- 1 501 staff 45 Jun 11 2007 index.html
Notice that we added the flag “i” to the “ls” command so we can see the inode number,
which identifies the files and directories on the filesystem in the first column.
Check the directory created by Docker:
sudo ls -lai $(docker volume inspect test-volume --format '{{ .Mountpoint }}')
256115 drwxr-xr-x 2 root root 4096 Jan 5 00:33 . 392833 drwx-----x 3 root root 4096 Jan 5 00:55 .. 256139 -rw-r--r-- 1 501 staff 45 Jun 11 2007 index.html
As you can see, only the parent directory is different, so we indeed see the same files
in the container and in the directory created by Docker.
Now let’s check our source directory.
Output:
total 12 256115 drwxr-xr-x 2 root root 4096 Jan 5 00:33 . 255512 drwxr-xr-x 11 ta ta 4096 Jan 5 00:32 .. 256139 -rw-r--r-- 1 501 staff 45 Jun 11 2007 index.html
Again, the same files, except the parent.
We confirmed, that we could create an empty volume directory,
we could populate it when we started a container and mounted the volume,
and the files appeared where Docker creates volumes. Now let’s check one more thing.
Since this is a special volume where we defined some parameters,
there is an opts.json right next to _data
sudo cat "$(dirname "$(docker volume inspect test-volume --format '{{ .Mountpoint }}')")"/opts.json
Output:
{"MountType":"none","MountOpts":"bind","MountDevice":"/home/ta/test-volume","Quota":{"Size":0}}
Now remove the test container:
docker container rm -f test-container
Check the directory created by Docker:
sudo ls -lai $(docker volume inspect test-volume --format '{{ .Mountpoint }}')
It is empty now.
392834 drwxr-xr-x 2 root root 4096 Jan 5 00:55 . 392833 drwx-----x 3 root root 4096 Jan 5 00:55 ..
And notice that even the inode has changed, not just the content disappeared.
On the other hand, the directory we created is untouched and you can still find the
index.html there.
Avoid accidental data loss on volumes
Let me show you an example using Docker Compose. The compose file would be the following:
volumes: docroot: driver: local driver_opts: type: none device: ./docroot o: bind services: httpd: image: httpd:2.4 volumes: - type: volume source: docroot target: /usr/local/apache2/htdocs
You can populate ./docroot in the project folder by running
You will then find index.html in the docroot folder.
You probably know that you can delete a compose project by running
docker compose down, and delete the volumes too by
passing the flag -v.
You can run it, and the volume will be destroyed, but not the content of the
already populated “docroot” folder. It happens, because the folder
which is managed by Docker in the Docker data root does not physically
have the content. So the one that was managed by Docker could be
safely removed, but it didn’t delete your data.
Docker CE volumes on Linux
This question seems to be already answered in the previous sections, but let’s
evaluate what we learned and add some more details.
So you can find the local default volumes under /var/lib/docker/volumes
if you didn’t change the data root.
For the sake of simplicity of the commands, I will keep using the default path.
The Docker data root is not accessible by normal users, only by administrators.
Run the following command:
sudo ls -la /var/lib/docker/volumes
You will see something like this:
total 140 drwx-----x 23 root root 4096 Jan 5 00:55 . drwx--x--- 13 root root 4096 Dec 10 14:27 .. drwx-----x 3 root root 4096 Jan 25 2023 0c5f9867e761f6df0d3ea9411434d607bb414a69a14b3f240f7bb0ffb85f0543 drwx-----x 3 root root 4096 Sep 19 13:15 1c963fb485fbbd5ce64c6513186f2bc30169322a63154c06600dd3037ba1749a ... drwx-----x 3 root root 4096 Jan 5 2023 apps_cache brw------- 1 root root 8, 1 Dec 10 14:27 backingFsBlockDev -rw------- 1 root root 65536 Jan 5 00:55 metadata.db
These are the names of the volumes and two additional special files.
-
backingFsBlockDev
-
metadata.db
We are not going to discuss it in more details. All you need to know at this point is
that this is where the volume folders are. Each folder has a sub-folder called “_data”
where the actual data is, and there could be an opts.json with metadata next to the
“_data” folder.
Note
When you use rootless Docker, the Docker data root will be in your user’s home.
$HOME/.local/share/docker
Docker Desktop volumes
Docker Desktop volumes are different depending on the operating system
and whether you want to run Linux containers or Windows containers.
Docker Desktop always runs a virtual machine
for Linux containers and runs Docker CE in it in a quite complicated way,
so your volumes will be in the virtual machine too. Because of that fact
when you want to access the volumes, you either have to find a way to run a shell
in the virtual machine, or find a way to share the filesystem on the network
and use your filebrowser, IDE or terminal on the host.
Parts of what I show here and more can be found in my presentation which
I gave on the 6th Docker Community All-Hands. Tyler Charboneau wrote a
blog post
about it, but you can also
find the video in the blog post.
Docker Desktop volumes on macOS
On macOS, you can only run Linux containers and there is no such thing as
macOS container yet (2024. january).
You can get to the volumes folder by running the following command:
docker run --rm -it --privileged --pid host ubuntu:22.04 \ nsenter --all -t 1 \ sh -c 'cd /var/lib/docker/volumes && sh'
Or just simply mount that folder to a container:
docker run --rm -it \ -v /var/lib/docker/volumes:/var/lib/docker/volumes \ --workdir /var/lib/docker/volumes \ ubuntu:22.04 \ bash
You can also run an NFS server in a container that mounts the volumes
so you can mount the remote fileshare on the host.
The following compose.yml file can be used to run the NFS server:
services: nfs-server: image: openebs/nfs-server-alpine:0.11.0 volumes: - /var/lib/docker/volumes:/mnt/nfs environment: SHARED_DIRECTORY: /mnt/nfs SYNC: sync FILEPERMISSIONS_UID: 0 FILEPERMISSIONS_GID: 0 FILEPERMISSIONS_MODE: "0755" privileged: true ports: - 127.0.0.1:2049:2049/tcp - 127.0.0.1:2049:2049/udp
Start the server:
Create the mount point on the host:
sudo mkdir -p /var/lib/docker/volumes sudo chmod 0700 /var/lib/docker
Mount the base directory of volumes:
sudo mount -o vers=4 -t nfs 127.0.0.1:/ /var/lib/docker/volumes
And list the content:
sudo ls -l /var/lib/docker/volumes
Docker Desktop volumes on Windows
Docker Desktop on Windows allows you to switch between Linux containers
and Windows containers.
To find out which one you are using,
run the following command:
docker info --format '{{ .OSType }}'
If it returns “windows”, you are using Windows containers, and if it returns
“linux”, you are using Linux containers.
Linux containers
Since Linux containers always require a virtual machine, you will have
your volumes in the virtual machine the same way as you would on macOS.
The difference is how you can access them. A common way is through
a Docker container. Usually I would run the following command.
docker run --rm -it --privileged --pid host ubuntu:22.04 ` nsenter --all -t 1 ` sh -c 'cd /var/lib/docker/volumes && sh'
But if you have an older kernel in WSL2 which doesn’t support the time namespace,
you can get an error message like:
nsenter: cannot open /proc/1/ns/time: No such file or directory
If that happens, make sure you have the latest kernel in WSL2.
If you built a custom kernel, you may need to rebuild it from a new
version.
If you can’t update the kernel yet, exclude the time namespace,
and run the following command:
docker run --rm -it --privileged --pid host ubuntu:22.04 ` nsenter -m -n -p -u -t 1 ` sh -c 'cd /var/lib/docker/volumes && sh'
You can simply mount the base directory in a container
the same way as we could on macOS:
docker run --rm -it ` -v /var/lib/docker/volumes:/var/lib/docker/volumes ` --workdir /var/lib/docker/volumes ` ubuntu:22.04 ` bash
We don’t need to run a server in a container to share the volumes,
since it works out of the box in WSL2. You can just open the Windows
explorer and go to
\\wsl.localhost\docker-desktop-data\data\docker\volumes
Warning
WSL2 let’s you edit files more easily even if the files are owned by root
on the volume, so do it at your own risk.
My recommendation is using it only for debugging.
Windows Containers
Windows containers can mount their volumes from the host.
Let’s create a volume
docker volume create windows-volume
Inspect the volume:
You will get something like this:
[ { "CreatedAt": "2024-01-06T16:27:03+01:00", "Driver": "local", "Labels": null, "Mountpoint": "C:\\ProgramData\\Docker\\volumes\\windows-volume\\_data", "Name": "windows-volume", "Options": null, "Scope": "local" } ]
So now you got the volume path on Windows in the “Mountpoint” field,
but you don’t have access to, it unless you are Administrator.
The following command works only from Powershell run as Administrator
cd $(docker volume inspect windows-volume --format '{{ .Mountpoint }}')
If you want to access it from Windows Explorer, you can first go to
Note
This folder is hidden by default, so if you want to open it, just type
the path manually in the navigation bar, or enable hidden folders
on Windows 11 (works differently on older Windows):
Menu bar » View » Show » Hidden Items
Then try to open the folder called “Docker” which gives you a prompt
to ask for permission to access to folder.
and then try to open the folder called “volumes”
which will do the same.
After that you can open any Windows container volume from Windows explorer.
Docker Desktop volumes on Linux
On Windows, you could have Linux containers and Window containers,
so you had to switch between them.
On Linux, you can install Docker CE in rootful and rootless mode,
and you can also install Docker Desktop. These are 3 different
and separate Docker installations and you can switch between them
by changing context or logging in as a different user.
You can check the existing contexts by running the following command:
If you have Docker CE installed on your Linux, and you are logged
in as a user who installed the rootless Docker,
and you also have Docker Desktop installed, you can see at least the
following three contexts:
NAME TYPE DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR default moby Current DOCKER_HOST based configuration unix:///var/run/docker.sock desktop-linux * moby Docker Desktop unix:///home/ta/.docker/desktop/docker.sock rootless moby Rootless mode unix:///run/user/1000/docker.sock
In order to use Docker Desktop, you need to switch to the context
called “desktop-linux”.
docker context use desktop-linux
Important
The default is usually rootful Docker CE and the other too are obvious.
Only the rootful Docker CE needs to run as root, so if you want to
interact with Docker Desktop, don’t make the mistake of running the docker commands
with sudo:
NAME TYPE DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR default * moby Current DOCKER_HOST based configuration unix:///var/run/docker.sock
In terms of accessing volumes, Docker Desktop works similarly on
macOS and Linux, so you have the following options:
Run a shell in the virtual machine using nsenter:
docker run --rm -it --privileged --pid host ubuntu:22.04 \ nsenter --all -t 1 \ sh -c 'cd /var/lib/docker/volumes && sh'
Or just simply mount that folder to a container:
docker run --rm -it \ -v /var/lib/docker/volumes:/var/lib/docker/volumes \ --workdir /var/lib/docker/volumes \ ubuntu:22.04 \ bash
And of course, you can use the nfs server compose project with
the following compose.yml
services: nfs-server: image: openebs/nfs-server-alpine:0.11.0 volumes: - /var/lib/docker/volumes:/mnt/nfs environment: SHARED_DIRECTORY: /mnt/nfs SYNC: sync FILEPERMISSIONS_UID: 0 FILEPERMISSIONS_GID: 0 FILEPERMISSIONS_MODE: "0755" privileged: true ports: - 127.0.0.1:2049:2049/tcp - 127.0.0.1:2049:2049/udp
and prepare the mount point. Remember, you can have Docker CE running as root,
which means /var/lib/docker probably exists, so let’s create the mount point
as /var/lib/docker-desktop/volumes:
sudo mkdir -p /var/lib/docker-desktop/volumes sudo chmod 0700 /var/lib/docker-desktop
And mount it:
sudo mount -o vers=4 -t nfs 127.0.0.1:/ /var/lib/docker-desktop/volumes
And check the content:
sudo ls -l /var/lib/docker-desktop/volumes
You could ask why we mount the volumes into a folder on the host,
which requires sudo if the docker commands don’t.
The reason is that you will need sudo to use the mount command,
so it shouldn’t be a problem to access the volumes as root.
Editing files on volumes
The danger of editing volume contents outside a container
Now you know how you can find out where the volumes are.
You also know how you can create a volume with a custom path,
even if you are using Docker Desktop, which creates the default
volumes inside a virtual machine.
But most of you wanted to know where the volumes were to edit the files.
Danger
Any operation inside the Docker data root is dangerous,
and can break your Docker completely, or cause problems
that you don’t immediately recognize, so you should never
edit files without mounting the volume into a container,
except if you defined a custom volume path
so you don’t have to go into the Docker data root.
Even if you defined a custom path, we are still talking about
a volume, which will be mounted into a container,
in which the files can be accessed by a process which
requires specific ownership and permissions. By editing
the files from the host, you can accidentally change the permission
or the owner making it inaccessible for the process in the container.
Even though I don’t recommend it, I understand that sometimes
we want to play with our environment to learn more about,
but we still have to try to find a less risky way to do it.
You know where the volumes are, and you can edit the files
with a text editor from command line or even from the graphical
interface. One problem on Linux and macOS could be setting
the proper permissions so you can edit the files even
if you are not root.
Discussing permissions could be another tutorial,
but this is one reason why we have to try to separate the data
managed by a process in a Docker container from the source code
or any files that requires an interactive user.
Just think of an application that is not running in a container,
but the files still have to be owned by another user.
An example could be a webserver, where the files has to be
owned by a user or group so the webserver has access to the files,
while you still should be able to upload files.
View and Edit files through Docker Desktop
Docker Desktop let’s you browse files from the GUI,
which is great for debugging, but I don’t recommend it for editing files,
even though Docker Desktop makes that possible too.
Let’s see why I am saying it.
Open the Containers tab of Docker Desktop.
Click on the three dots in the line of the container in which you want to browse files
Go to a file that you want to edit
Note
Notice that Docker Desktop shows you whether the files are modified
on the container’s filesystem, or you see a file on a volume.
Right click on the file and select “Edit file”.
Before you do anything, run a test container:
docker run -d --name httpd -v httpd_docroot:/usr/local/apache2/htdocs httpd:2.4
And check the permissions of the index file:
docker exec -it httpd ls -l /usr/local/apache2/htdocs/
You will see this:
-rw-r--r-- 1 504 staff 45 Jun 11 2007 index.html
You can then edit the file and click on the floppy icon on the right side or just
press CTRL+S (Command+S on macOS) to save the modification.
Then run the following command from a terminal:
docker exec -it httpd ls -l /usr/local/apache2/htdocs/
And you will see that the owner of the file was changed to root.
total 4 -rw-r--r-- 1 root root 69 Jan 7 12:21 index.html
One day it might work better, but I generally don’t recommend
editing files in containers from the Graphical interface.
Edit only source code that you mount into the container during development
or Use Compose watch to update the files when you edit them,
but let the data be handled only by the processes in the containers.
Some applications are not optimized for running in containers and there are
different folders and files at the same place where the code is,
so it is hard to work with volumes and mounts while you let the process
in the container change a config file, which you also want to edit occasionally.
In that case you ned to learn how permissions are handled on Linux
using the chmod and chown commands so you both
have permission to access the files.
Container based dev environments
Docker Desktop Dev environment
One of the features of Docker Desktop is that you can run a development
environment in a container. In this tutorial we will not discuss it in
details, but it is good to know that it exists, and you can
basically work inside a container into which you can mount volumes.
More information in the documentation of the Dev environment
Visual Studio Code remote development
The dev environment of Docker Desktop can be opened from Visual Studio Code
as it supports opening projects in containers similarly to how it supports
remote development through SSH connection or in Windows Subsystem for Linux.
You can use it without Docker Desktop to simply open a shell in a container
or even open a project in a container.
More information is in the documentation of VSCode about containers.
Visual Studio Code dev containers
Microsoft also created container images for creating a dev container,
which is similar to what Docker Desktop supports, but the process
of creating a dev container is different.
More information in the documentation of VSCode about dev containers.
Conclusion
There are multiple ways to browse the content of the Docker volumes,
but it is not recommended to edit the files on the volumes.
If you know enough about how containers work and what are the folders
and files that you can edit without harming your system,
you probably know enough not to edit the files that way in the first place.
For debugging reasons or to learn about Docker by changing things
in the environment, you can still edit the files at your own risk.
Everything I described in this tutorial is true even if the user is not
an interactive user, but an external user from the container’s point of view,
trying to manage files directly in the Docker data root.
So with that in mind if you ever think of doing something like that,
stop for a moment, grab a paper and write the following sentence
20 times to the paper:
“I do not touch the Docker data root directly.”
If you enjoyed this tutorial, I also recommend reading about
Volume-only Compose projects.
Update! From Windows 1809 onwards this is no longer an issue!
See 6 Things You Can Do with Docker in Windows Server 2019 That You Couldn’t Do in Windows Server 2016
You use Docker volumes to store state outside of containers, so your data survives when you replace the container to update your app. Docker uses symbolic links to give the volume a friendly path inside the container, like C:\data. Some application runtimes try to follow the friendly path to the real location — which is actually outside the container — and get themselves into trouble.
This issue may not affect all application runtimes, but I have seen it with Windows Docker containers running Java, Node JS, Go, PHP and .NET Framework apps. So it’s pretty widespread.
You can avoid that issue by using a mapped drive (say G:\) inside the container. Your app writes to the G drive and the runtime happily lets the Windows filesystem take care of actually finding the location, which happens to be a symlink to a directory on the Docker host.
Filesystems in Docker Containers
An application running in a container sees a complete filesystem, and the process can read and write any files it has access to. In a Windows Docker container the filesystem consists of a single C drive, and you’ll see all the usual file paths in there — like C:\Program Files and C:\inetpub. In reality the C drive is composed of many parts, which Docker assembles into a virtual filesystem.
It’s important to understand this. It’s the basis for how images are shared between multiple containers, and it’s the reason why data stored in a container is lost when the container is removed. The virtual filesystem the container sees is built up of many image layers which are read-only and shared, and a final writeable layer which is unique to the container:
When processes inside the container modify files from read-only layers, they’re actually copied into the writeable layer. That layer stores the modified version and hides the original. The underlying file in the read-only layer is unchanged, so images don’t get modified when containers make changes.
Removing a container removes its writeable layer and all the data in it, so that’s not the place to store data if you run a stateful application in a container. You can store state in a volume, which is a separate storage location that one or more containers can access, and has a separate lifecycle to the container:
Storing State in Docker Volumes
Using volumes is how you store data in a Dockerized application, so it survives beyond the life of a container. You run your database container with a volume for the data files. When you replace your container from a new image (to deploy a Windows update or a schema change), you use the same volume, and the new container has all the data from the original container.
The SQL Server Docker lab on GitHub walks you through an example of this.
You define volumes in the Dockerfile, specifying the destination path where the volume is presented to the container. Here’s a simple example which stores IIS logs in a volume:
#escape=`
FROM microsoft/iis
VOLUME C:\inetpub\logs
You can build an image from that Dockerfile and run it in a container. When you run docker container inspect you will see that there is a mount point listed for the volume:
"Mounts": [
{
"Type": "volume",
"Name": "cfc1ab55dbf6e925a1705673ff9f202d0ee2157dcd199c02111813b05ddddf22",
"Source": "C:\\ProgramData\\docker\\volumes\\cfc1ab55dbf6e925a1705673ff9f202d0ee2157dcd199c02111813b05ddddf22\\_data",
"Destination": "C:\\inetpub\\logs",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
]
The source location of the mount shows the physical path on the Docker host where the files for the volume are written — in C:\ProgramData\docker\volumes. When IIS writes logs from the container in C:\Inetpub\logs, they’re actually written to the directory in C:\ProgramData\docker\volumes on the host.
The destination path for a volume must be a new folder, or an existing empty folder. Docker on Windows is different from linux in that respect, you can’t use a destination folder which already contains data from the image, and you can’t use a single file as a destination.
Docker surfaces the destination directory for the volume as a symbolic link ( symlink ) inside the container, and that’s where the trouble begins.
Symlink Directories
Symbolic links have been a part of the Windows filesystem for a long time, but they’re nowhere near as popluar as they are in Linux. A symlink is just like an alias, which abstracts the physical location of a file or directory. Like all abstractions, it lets you work at a higher level and ignore the implementation details.
In Linux it’s common to install software to a folder which contains the version name — like /opt/hbase-1.2.3 and then create a sylmink to that directory, with a name that removes the version number — /opt/hbase. In all your scripts and shortcuts you use the symlink. When you upgrade the software, you change the symlink to point to the new version and you don’t need to change anything else. You can also leave the old version in place and rollback by changing the symlink.
You can do the same in Windows, but it’s much less common. The symlink mechanism is how Docker volumes work in Windows. If you docker container exec into a running container and look at the volume directory, you’ll it listed as a symlink directory (SYMLINKD) with a strange path:
c:\>dir C:\inetpub
Volume in drive C has no label.
Volume Serial Number is 90D3-C0CE
Directory of C:\inetpub
06/30/2017 10:08 AM <DIR> .
06/30/2017 10:08 AM <DIR> ..
01/18/2017 07:43 PM <DIR> custerr
01/18/2017 07:43 PM <DIR> history
01/18/2017 07:43 PM <SYMLINKD> logs [\\?\ContainerMappedDirectories\8305589A-2E5D...]
06/30/2017 10:08 AM <DIR> temp
01/18/2017 07:43 PM <DIR> wwwroot
The logs directory is actually a symlink directory, and it points to the path \\?\ContainerMappedDirectories\8305589A-2E5D... The Windows filesystem understands that symlink, so if apps write directly to the logs folder, Windows writes to the symlink directory, which is actually the Docker volume on the host.
The trouble really begins when you configure your app to use a volume, and the application runtime tries to follow the symlink. Runtimes like Go, Java, PHP, NodeJS and even .NET will do this — they resolve the symlink to get the real directory and try to write to the real path. When the “real” path starts with \\?\ContainerMappedDirectories\, the runtime can’t work with it and the write fails. It might raise an exception, or it might just silently fail to write data. Neither of which is much good for your stateful app.
DOS Devices to the Rescue
The solution — as always — is to introduce another layer of abstraction, so the app runtime doesn’t directly use the symlink directory. In the Dockerfile you can create a drive mapping to the volume directory, and configure the app to write to the drive. The runtime just sees a drive as the target and doesn’t try to do anything special — it writes the data, and Windows takes care of putting it in the right place.
I use the G drive in my Dockerfiles, just to distance it from the C drive. Ordinarily you use the subst utility to create a mapped drive, but that doesn’t create a map which persists between sessions. Instead you need to write a registry entry in your Dockerfile to permanently set up the mapped drive:
VOLUME C:\data
RUN Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\DOS Devices' -Name 'G:' -Value "\??\C:\data" -Type String;
This creates a fake G drive which maps to the volume directory C:\data. Then you can configure your app to write to the G drive and it won’t realise the target is a symlink, so it won’t try to resolve the path and it will write correctly.
I use this technique in these Jenkins and Bonobo Dockerfiles, where I also set up the
Gdrive as the target in the app configuration.
How you configure the storage target depends on the app. Jenkins uses an environment variable, which is very easy. Bonobo uses Web.config, which means running some XML updates with PowerShell in the Dockerfile. This technique means you need to mentally map the fake G drive to a real Docker volume, but it works with all the apps I’ve tried, and it also works with volume mounts.
Mounting Volumes
Docker volumes on Windows are always created in the path of the graph driver, which is where Docker stores all image layers, writeable container layers and volumes. By default the root of the graph driver in Windows is C:\ProgramData\docker, but you can mount a volume to a specific directory when you run a container.
I have a server with a single SSD for the C drive, which is where my Docker graph is stored. I get fast access to image layers at the cost of zero redundancy, but that’s fine because I can always pull images again if the disk fails. For my application data, I want to use the E drive which is a RAID array of larger but slower spinning disks.
When I run my local Git server and Jenkins server in Docker containers I use a volume mount, pointing the Docker volume in the container to a location on my RAID array:
docker container run -v E:\bonobo:C:\data sixeyed/bonobo
Actually I use a compose file for my services, but that’s for a different post.
So now there are multiple mappings from the G drive the app uses to the Docker volume, and the underlying storage location:
G:\ -> C:\data -> \\?\ContainerMappedDirectories\xyz -> E:\bonobo
Book Plug
I cover volumes — and everything else to do with Docker on Windows — in my book Docker on Windows, which is out now.
If you’re not into technical books, all the code samples are on GitHub: sixeyed/docker-on-windows and every sample has a Docker image on the Hub: dockeronwindows.
Use the G Drive For Now
I’ve hit this problem with lots of different app runtimes, so I’ve started to do this as the norm with stateful applications. It saves a lot of time to configure the G drive first, and ensure the app is writing state to the G drive, instead of chasing down issues later.
The root of the problem actually seems to be a change in the file descriptor for symlink directories in Windows Server 2016. Issues have been logged with some of the application runtimes to work correctly with the symlink (like in Go and in Java), but until they’re fixed the G drive solution is the most robust that I’ve found.
It would be nice if the image format supported this, so you could write VOLUME G: in the Dockerfile and hide all this away. But this is a Windows-specific issue and Docker is a platform that works in the same way across multiple operating systems. Drive letters don’t mean anything in Linux so I suspect we’ll need to use this workaround for a while.
