[FIXED] How to process pathnames with ! within an for /F loop?

Issue

In a complex batch file I want to read in files with paths, among other things, to read them into a variable one after the other separated by spaces.

This works with the following code so far quite well - but only if the path does not contain an exclamation mark.

Even using the setlocal command (enabledelayedexpansion / disabledelayedexpansion) I did not succeed in processing exclamation marks.

Does anyone here have a clever idea to the problem?

The following example batch creates a text file in the current directory and then reads it in a for /F loop. At the end all three paths from the text file should be in the variable %Output%. But with the exclamation mark.

@echo off

setlocal enabledelayedexpansion

echo This is an example^^! > "textfile.txt"
echo This is a second example^^! >> "textfile.txt"
echo And this line have an ^^! exclamation mark in the middle >> "textfile.txt"

for /F "usebackq tokens=* delims=" %%a in (textfile.txt) do (
    set "Record=%%a"
    set "Output=!Output!!Record! - "
  )
)

echo %Output%
echo !Output!

endlocal

The Output is like this:

This is an example  - This is a second example  - And this line have an  exclamation mark in the middle

But should be like this:

This is an example!  - This is a second example!  - And this line have an ! exclamation mark in the middle

Solution

It is advisable not using delayed variable expansion on processing files and directories, lines in a text file, strings not defined by the batch file itself, or output captured from the execution of a program or a command line. If it is for some reasons necessary to make use of delayed variable expansion inside a FOR loop, there should be first assigned the file/directory name, the line, or the string to process to an environment variable while delayed expansion is disabled and then enable delayed expansion temporary inside the FOR loop.

Here is a batch file demo which can be simply run from within a command prompt window or by double clicking on the batch file. It creates several files for demonstration in the directory for temporary files, but deletes them all before exiting.

@echo off
setlocal EnableExtensions DisableDelayedExpansion

echo This is an example!> "%TEMP%\TextFile.tmp"
echo This is a second example!>> "%TEMP%\TextFile.tmp"
echo And this line has an exclamation mark ! in the middle.>> "%TEMP%\TextFile.tmp"

set "Output="
(for /F usebackq^ delims^=^ eol^= %%I in ("%TEMP%\TextFile.tmp") do set "Line=%%I" & call :ConcatenateLines) & goto ContinueDemo
:ConcatenateLines
set "Output=%Output% - %Line%" & goto :EOF

:ContinueDemo
cls
echo/
echo All lines concatenated are:
echo/
echo %Output:~3%
set "Output="
del "%TEMP%\TextFile.tmp"

echo File with name ".Linux hidden file!">"%TEMP%\.Linux hidden file!"
echo File with name "A simple test!">"%TEMP%\A simple test!"
echo File with name "  100%% Development & 'Test' (!).tmp">"%TEMP%\  100%% Development & 'Test(!)'.tmp"
echo/
echo Files with ! are:
echo/
for /F "eol=| tokens=* delims=" %%I in ('dir "%TEMP%\*!*" /A-D /B /ON 2^>nul') do (
    set "NameFile=%%I"
    set "FileName=%%~nI"
    set "FileExtension=%%~xI"
    set "FullName=%TEMP%\%%I"
    setlocal EnableDelayedExpansion
    if defined FileName (
        if defined FileExtension (
            echo File with ext. !FileExtension:~1!: !NameFile!
        ) else (
            echo Extensionless file: !NameFile!
        )
    ) else echo Extensionless file: !NameFile!
    del "!FullName!"
    endlocal
)
endlocal
echo/
@setlocal EnableExtensions EnableDelayedExpansion & for /F "tokens=1,2" %%G in ("!CMDCMDLINE!") do @endlocal & if /I "%%~nG" == "cmd" if /I "%%~H" == "/c" set /P "=Press any key to exit the demo . . . "<nul & pause >nul

The output of this batch file is:

All lines concatenated are:

This is an example! - This is a second example! - And this line has an exclamation mark ! in the middle.

Files with ! are:

File with ext. tmp:   100% Development & 'Test(!)'.tmp
Extensionless file: .Linux hidden file!
Extensionless file: A simple test!

The text file example with concatenating lines makes use of a subroutine called from within the FOR loop processing the lines in the text file. The syntax used here is for maximum performance by getting the subroutine as near as possible to the FOR command line. That is important if the FOR loop has to process hundreds or even thousands of items.

The example processing file names enables and disables delayed expansion inside the FOR loop after having assigned all parts of the currently processed file to environment variables. It could be useful to reduce the list of environment variables before processing thousands of files for a better performance on using this method.

Another method is shown in Magoo´s answer using the command CALL to get a command line with referenced environment variables (re)defined inside the loop parsed a second time. I used that method also in the past quite often, but don't that anymore as it is not fail-safe and not efficient. call set results in searching by cmd.exe in current directory and next in all directories of environment variable PATH for a file with name set and a file extension of environment variable PATHEXT. So it results in lots of file system accesses in the background on each iteration of the FOR loop and if there is by chance a file set.exe, set.bat, set.cmd, etc. found by cmd.exe somewhere, the batch file does not work anymore as expected because of running the executable or calling the batch file instead of the (re)definition of the environment variable.

The following answers written by me could be also helpful:

• How to read and print contents of text file line by line?
  It explains in full details how to process all lines of a text file.
• How to pass environment variables as parameters by reference to another batch file?
  It explains in full details what the commands SETLOCAL and ENDLOCAL do.
• How to pass a command that may contain special characters (such as % or !) inside a variable to a for /f loop?
  This is an example of a batch file designed to process video files with any valid file name on any Windows computer very efficient, safe and secure with full explanation.

Answered By - Mofi

Answer Checked By - Mary Flores (PHPFixing Volunteer)