[FIXED] How to convert piped/awk output to string/variable

Issue

I'm trying to create a bash function that automatically updates a cli tool. So far I've managed to get this:

update_cli_tool () {
    # the following will automatically be redirected to .../releases/tag/vX.X.X
    # there I get the location from the header, and remove it to get the full url
    latest_release_url=$(curl -i https://github.com/.../releases/latest | grep location: | awk -F 'location: ' '{print $2}')
    # to get the version, I get the 8th element from the url .../releases/tag/vX.X.X
    latest_release_version=$(echo "$latest_release_url" | awk -F '/' '{print 8}')
    
    # this is where it breaks
    # the first part just replaces the "tag" with "download" in the url
    full_url="${latest_release_url/tag/download}/.../${latest_release_version}.zip"
    echo "$full_url"  # or curl $full_url, also fails
}

Expected output: https://github.com/.../download/vX.X.X/vX.X.X.zip

Actual output: -.zip-.../.../releases/download/vX.X.X

When I just echo "latest_release_url: $latest_release_url" (same for version), it prints it correctly, but not when I use the above mentioned flow. When I hardcode the ..._url and ..._version, the full_url works fine. So my guess is I have to somehow capture the output and convert it to a string? Or perhaps concatenate it another way?

Note: I've also used ..._url=`curl -i ...` (with backticks instead of $(...)), but this gave me the same results.

Solution

The curl output will use \r\n line endings. The stray carriage return in the url variable is tripping you up. Observe it with printf '%q\n' "$latest_release_url"

Try this:

latest_release_url=$(
    curl --silent -i https://github.com/.../releases/latest \
    | awk -v RS='\r\n' '$1 == "location:" {print $2}'
)

Then the rest of the script should look right.

Answered By - glenn jackman

Answer Checked By - Pedro (PHPFixing Volunteer)

Read More

[FIXED] How do I send keys to an active window in Powershell?

Issue

I want my script to open an application and send keys to it after it's opened. Currently, if I run the script, it opens the app but does not send the keys.

If I run just the send keys after the app has already been opened, it works.

Here is what I've got so far:

Start-Process -FilePath "C:\Program Files\VMware\VMware Horizon View Client\vmware-view.exe" -Wait -WindowStyle Normal

$wshell = New-Object -ComObject wscript.shell;
$wshell.AppActivate('VMware')
Sleep 1
$wshell.SendKeys('~')
Sleep 3
$wshell.SendKeys('username')
Sleep 2
$wshell.SendKeys('{TAB}')
Sleep 1
$wshell.SendKeys('password')

Solution

By using -Wait with Start-Process, you're blocking the call until the launched process terminates.

Thus, your attempts to send keystrokes will invariably fail, because they'll be sent after the target program has terminated.

---

Therefore:

• Don't use -Wait

• Use -PassThru, which makes Start-Process emit a process-information object representing the newly launched process, whose .ID property contains the PID (process ID).

• For more reliable targeting, you can pass a PID to $wshell.AppActivate()

• The general caveat applies: sending keystrokes, i.e. simulating user input is inherently unreliable.

$ps = Start-Process -PassThru -FilePath "C:\Program Files\VMware\VMware Horizon View Client\vmware-view.exe" -WindowStyle Normal

$wshell = New-Object -ComObject wscript.shell

# Wait until activating the target process succeeds.
# Note: You may want to implement a timeout here.
while (-not $wshell.AppActivate($ps.Id)) {
  Start-Sleep -MilliSeconds 200
}

$wshell.SendKeys('~')
Sleep 3
$wshell.SendKeys('username')
Sleep 2
$wshell.SendKeys('{TAB}')
Sleep 1
$wshell.SendKeys('password')

Answered By - mklement0

Answer Checked By - Senaida (PHPFixing Volunteer)

Read More

[FIXED] How to sort a list of different filenames sorting a number in the filename with bash to find the "newest" filename in bash?

Issue

I run a command, getFiles, that outputs a newline delimited list filenames. Very similar to the output of ls -1. The filenames all end in a number representing a kind of timestamp. Larger numbers are more recent. The filenames follow no pattern. e.g. file1234, other44234, something34142, carrot123.

I need to find the filename with the largest number (numerically). In this example other44234.

After finding the filename I need to pass it as an argument into another command. i.e. doItToIt "$THE_FILE"

Solution

Another way:

$ sed -E 's/^([a-Z]+)([0-9]+)/\1 \2/' files.txt |
    sort -n -k2 |
    tail -n1 |
    tr -d ' '

other44234

Answered By - Gilles Quenot

Answer Checked By - Dawn Plyler (PHPFixing Volunteer)

Read More

[FIXED] Why equal to operator does not work if it is surrounded by space?

Issue

I tried the following script

#!/bin/bash
var1="Test 1" 
var2="Test 2"
if [ "$var1"="$var2" ] 
  then 
    echo "Equal" 
  else 
    echo "Not equal"
fi

It gave me Equal. Although it should have printed Not equal

Only when I inserted space around = it worked as intended

if [ "$var1" = "$var2" ] 

and printed Not equal

Why is it so? Why "$var1"="$var2" is not same as "$var1" = "$var2"?

Moreover, when I wrote if [ "$var1"= "$var2" ], it gave

line 4: [: Test 1=: unary operator expected

What does it it mean? How come its expecting unary operator?

Solution

test (or [ expr ]) is a builtin function. Like all functions in bash, you pass its arguments as whitespace separated words.

As the man page for bash builtins states: "Each operator and operand must be a separate argument."

It's just the way bash and most other Unix shells work.

Variable assignment is different.

In bash a variable assignment has the syntax: name=[value]. You cannot put unquoted spaces around the = because bash would not interpret this as the assignment you intend. bash treats most lists of words as a command with parameters.

E.g.

# call the command or function 'abc' with '=def' as argument
abc =def

# call 'def' with the variable 'abc' set to the empty string
abc= def

# call 'ghi' with 'abc' set to 'def'
abc=def ghi

# set 'abc' to 'def ghi'
abc="def ghi"

Answered By - CB Bailey

Answer Checked By - Mildred Charles (PHPFixing Admin)

Read More

[FIXED] How can I remove the first line of a text file using bash/sed script?

Issue

I need to repeatedly remove the first line from a huge text file using a bash script.

Right now I am using sed -i -e "1d" $FILE - but it takes around a minute to do the deletion.

Is there a more efficient way to accomplish this?

Solution

Try tail:

tail -n +2 "$FILE"

-n x: Just print the last x lines. tail -n 5 would give you the last 5 lines of the input. The + sign kind of inverts the argument and make tail print anything but the first x-1 lines. tail -n +1 would print the whole file, tail -n +2 everything but the first line, etc.

GNU tail is much faster than sed. tail is also available on BSD and the -n +2 flag is consistent across both tools. Check the FreeBSD or OS X man pages for more.

The BSD version can be much slower than sed, though. I wonder how they managed that; tail should just read a file line by line while sed does pretty complex operations involving interpreting a script, applying regular expressions and the like.

Note: You may be tempted to use

# THIS WILL GIVE YOU AN EMPTY FILE!
tail -n +2 "$FILE" > "$FILE"

but this will give you an empty file. The reason is that the redirection (>) happens before tail is invoked by the shell:

1. Shell truncates file $FILE
2. Shell creates a new process for tail
3. Shell redirects stdout of the tail process to $FILE
4. tail reads from the now empty $FILE

If you want to remove the first line inside the file, you should use:

tail -n +2 "$FILE" > "$FILE.tmp" && mv "$FILE.tmp" "$FILE"

The && will make sure that the file doesn't get overwritten when there is a problem.

Answered By - Aaron Digulla

Answer Checked By - Clifford M. (PHPFixing Volunteer)

Read More

[FIXED] How to get a password from a shell script without echoing

Issue

I have a script that automates a process that needs access to a password protected system. The system is accessed via a command-line program that accepts the user password as an argument.

I would like to prompt the user to type in their password, assign it to a shell variable, and then use that variable to construct the command line of the accessing program (which will of course produce stream output that I will process).

I am a reasonably competent shell programmer in Bourne/Bash, but I don't know how to accept the user input without having it echo to the terminal (or maybe having it echoed using '*' characters).

Can anyone help with this?

Solution

Here is another way to do it:

#!/bin/bash
# Read Password
echo -n Password: 
read -s password
echo
# Run Command
echo $password

The read -s will turn off echo for you. Just replace the echo on the last line with the command you want to run.

In some shells (e.g. bash) read supports -p prompt-string which will allow the echo and read commands to be combined.

read -s -p "Password: " password

Answered By - wsware

Answer Checked By - Gilberto Lyons (PHPFixing Admin)

Read More

[FIXED] How to create a spherical angular gradient around y-axis using Lua in Filter Forge?

Issue

I am working on a Spherical Map Script that generates a linear gradient along the y-axis and an angular gradient around the y-axis in the green and red channels, respectively. I have been unable to find any documentation suited to this online, but have experimented using examples in Javascript and C#. The linear gradient has worked out fine, but the angular gradient (one describing a 360 degree arc around the y-axis) continues to elude me.

My working script is as follows.

-- 3d sphere 
    -- produces rgb gradudient mapped to a 3d sphere, but not correctly. 
    -- this is basically missing an angle gradient around the y-axis...
function prepare()
    -- tilt & rotation precalc
    toRad = 180/math.pi
    -- toCir = 360/math.p -- may or may not work for circumference...

    radius = get_slider_input(RADIUS)

    angle = get_angle_input(ROTATION)/toRad
    cosa = math.cos(angle)
    sina = math.sin(angle)

    tilt = get_angle_input(TILT)/toRad
    cosa2 = math.cos(tilt)
    sina2 = math.sin(tilt)
end;


function get_sample(x, y)
    local r, g, b, a = get_sample_map(x, y, SOURCE)
    -- color gradient example
    --  local r = x
    --  local g = y
    --  local b = (x + y) / 2
    --  local a = 1
    -- spherical mapping formulae (polar to cartesian, apparently)
    --  local x = x * math.pi -- * aspect 
    --  local y = y * math.pi
    --  local nx = math.cos(x) * math.sin(y) 
    --  local ny = math.sin(x) * math.sin(y) 
    --  local nz = math.cos(y) 
    -- cartesian to polar (reference)
    --  example 1
    --  r = math.sqrt(((x * x) + (y * y) + (z * z))) 
    --  long = math.acos(x / math.sqrt((x * x) + (y * y))) * (y < 0 ? -1 : 1) 
    --  lat = math.acos(z / radius) * (z < 0 ? -1 : 1) 
    --  example 2
    --      r = math.sqrt((x * x) + (y * y) + (z * z)) 
    --      long = math.acos(x / math.sqrt((x * x) + (y * y))) * (y < 0 ? -1 : 1) 
    --  lat = math.acos(z / r) 
            -- equations cannot accept boolean comparison
            -- boolean syntax may not be valid in lua
   
    -- image generation
    -- shift origin to center and set radius limits
    local px = (x*2.0) - 1.0
    local py = (y*2.0) - 1.0
    px = px/radius
    py = py/radius
    local len = math.sqrt((px*px)+(py*py))
    if len > 1.0 then return 0,0,0,0 end

    local z = -math.sqrt(1.0 - ((px*px)+(py*py)))

    -- cartesian to polar
        -- r = math.sqrt((x * x) + (y * y) + (z * z))
        -- lat = math.acos(z / r)
        -- long = math.acos(x / math.sqrt((x * x) + (y * y))) * (y < 0) ? -1 : 1) 

    -- apply rotaqtion and tilt (order is important)
    local tz = (cosa2 * z) - (sina2 * py) 
    local ty = (sina2 * z) + (cosa2 * py) -- gradient along y-axis is correct
    z = tz 
    py = ty

    local tx = (cosa * px) - (sina * z) -- gradient needs to go around y-axis
    local tz = (sina * px) + (cosa * z)
    px = tx
    z = tz

        -- r = math.sqrt((x * x) + (y * y) + (z * z)) 
        -- lat = math.acos(z / r) -- invalid z for this; what is correct source?
        -- long = math.acos(x / math.sqrt((x * x) + (y * y))) -- map -1 : 1 needed
        -- long = (sina * px) + (cosa * z) -- ok;  2 full rotations

    -- return r, g, b, a
    -- return px,py,z,a 
    return px/2+.5,py/2+.5,z/2+.5,a 
    -- return px/2+.5,px/2+.5,px/2+.5,a 
    -- return long,long,long,a 
    -- return px/2+.5,py/2+.5,long,a 
end;

Solution

I found a way to construct a spherical angular gradient from the hemispherical axis gradients I started with, by using three as the RBG channels in an RBG to HSL conversion. I've used this to map textures to spheres in Filter Forge in map scripts like this.

I thought I'd share the solution for anyone who's followed my question here.

Answered By - DavidBryanRoberson

Answer Checked By - David Goodson (PHPFixing Volunteer)

Read More

[FIXED] How do I parse command line arguments in Bash?

Issue

Say, I have a script that gets called with this line:

./myscript -vfd ./foo/bar/someFile -o /fizz/someOtherFile

or this one:

./myscript -v -f -d -o /fizz/someOtherFile ./foo/bar/someFile 

What's the accepted way of parsing this such that in each case (or some combination of the two) $v, $f, and $d will all be set to true and $outFile will be equal to /fizz/someOtherFile?

Solution

Bash Space-Separated (e.g., --option argument)

cat >/tmp/demo-space-separated.sh <<'EOF'
#!/bin/bash

POSITIONAL_ARGS=()

while [[ $# -gt 0 ]]; do
  case $1 in
    -e|--extension)
      EXTENSION="$2"
      shift # past argument
      shift # past value
      ;;
    -s|--searchpath)
      SEARCHPATH="$2"
      shift # past argument
      shift # past value
      ;;
    --default)
      DEFAULT=YES
      shift # past argument
      ;;
    -*|--*)
      echo "Unknown option $1"
      exit 1
      ;;
    *)
      POSITIONAL_ARGS+=("$1") # save positional arg
      shift # past argument
      ;;
  esac
done

set -- "${POSITIONAL_ARGS[@]}" # restore positional parameters

echo "FILE EXTENSION  = ${EXTENSION}"
echo "SEARCH PATH     = ${SEARCHPATH}"
echo "DEFAULT         = ${DEFAULT}"
echo "Number files in SEARCH PATH with EXTENSION:" $(ls -1 "${SEARCHPATH}"/*."${EXTENSION}" | wc -l)

if [[ -n $1 ]]; then
    echo "Last line of file specified as non-opt/last argument:"
    tail -1 "$1"
fi
EOF

chmod +x /tmp/demo-space-separated.sh

/tmp/demo-space-separated.sh -e conf -s /etc /etc/hosts

Output from copy-pasting the block above

FILE EXTENSION  = conf
SEARCH PATH     = /etc
DEFAULT         =
Number files in SEARCH PATH with EXTENSION: 14
Last line of file specified as non-opt/last argument:
#93.184.216.34    example.com

Usage

demo-space-separated.sh -e conf -s /etc /etc/hosts

---

Bash Equals-Separated (e.g., --option=argument)

cat >/tmp/demo-equals-separated.sh <<'EOF'
#!/bin/bash

for i in "$@"; do
  case $i in
    -e=*|--extension=*)
      EXTENSION="${i#*=}"
      shift # past argument=value
      ;;
    -s=*|--searchpath=*)
      SEARCHPATH="${i#*=}"
      shift # past argument=value
      ;;
    --default)
      DEFAULT=YES
      shift # past argument with no value
      ;;
    -*|--*)
      echo "Unknown option $i"
      exit 1
      ;;
    *)
      ;;
  esac
done

echo "FILE EXTENSION  = ${EXTENSION}"
echo "SEARCH PATH     = ${SEARCHPATH}"
echo "DEFAULT         = ${DEFAULT}"
echo "Number files in SEARCH PATH with EXTENSION:" $(ls -1 "${SEARCHPATH}"/*."${EXTENSION}" | wc -l)

if [[ -n $1 ]]; then
    echo "Last line of file specified as non-opt/last argument:"
    tail -1 $1
fi
EOF

chmod +x /tmp/demo-equals-separated.sh

/tmp/demo-equals-separated.sh -e=conf -s=/etc /etc/hosts

Output from copy-pasting the block above

FILE EXTENSION  = conf
SEARCH PATH     = /etc
DEFAULT         =
Number files in SEARCH PATH with EXTENSION: 14
Last line of file specified as non-opt/last argument:
#93.184.216.34    example.com

Usage

demo-equals-separated.sh -e=conf -s=/etc /etc/hosts

---

To better understand ${i#*=} search for "Substring Removal" in this guide. It is functionally equivalent to `sed 's/[^=]*=//' <<< "$i"` which calls a needless subprocess or `echo "$i" | sed 's/[^=]*=//'` which calls two needless subprocesses.

---

Using bash with getopt[s]

getopt(1) limitations (older, relatively-recent getopt versions):

• can't handle arguments that are empty strings
• can't handle arguments with embedded whitespace

More recent getopt versions don't have these limitations. For more information, see these docs.

---

POSIX getopts

Additionally, the POSIX shell and others offer getopts which doen't have these limitations. I've included a simplistic getopts example.

cat >/tmp/demo-getopts.sh <<'EOF'
#!/bin/sh

# A POSIX variable
OPTIND=1         # Reset in case getopts has been used previously in the shell.

# Initialize our own variables:
output_file=""
verbose=0

while getopts "h?vf:" opt; do
  case "$opt" in
    h|\?)
      show_help
      exit 0
      ;;
    v)  verbose=1
      ;;
    f)  output_file=$OPTARG
      ;;
  esac
done

shift $((OPTIND-1))

[ "${1:-}" = "--" ] && shift

echo "verbose=$verbose, output_file='$output_file', Leftovers: $@"
EOF

chmod +x /tmp/demo-getopts.sh

/tmp/demo-getopts.sh -vf /etc/hosts foo bar

Output from copy-pasting the block above

verbose=1, output_file='/etc/hosts', Leftovers: foo bar

Usage

demo-getopts.sh -vf /etc/hosts foo bar

The advantages of getopts are:

1. It's more portable, and will work in other shells like dash.
2. It can handle multiple single options like -vf filename in the typical Unix way, automatically.

The disadvantage of getopts is that it can only handle short options (-h, not --help) without additional code.

There is a getopts tutorial which explains what all of the syntax and variables mean. In bash, there is also help getopts, which might be informative.

Answered By - Bruno Bronosky

Answer Checked By - Marilyn (PHPFixing Volunteer)

Read More

[FIXED] how to get multiple arguments from option in bash script

Issue

#parse options
while getopts ":d:b:n:" opt; do
  case $opt in
    d)
          DIRS+=("$OPTARG")
          echo $DIRS
          ;;
    b)
          PATHBACKUP=$OPTARG
          echo $PATHBACKUP
          ;;
    n)
          FNAME=$OPTARG
          echo $FNAME
          ;;
    :)
          echo "Option -$OPTARG requires an argument." >&2
          error
          exit 1
          ;;
  esac
done
shift $(( OPTIND - 1 ))

This is my code I am trying to store every argument after -d to $DIRS However, when I echo $DIRS I only get the first argument

Example:

/.script -d /dev /home/work -b /backup
echo $DIRS
echo $PATHBACKUP
> /dev
> /backup

Solution

Use multiple -ds

./script -d /dev -d /home/work -b /backup

otherwise, the first non-option (i.e. /home/work) would stop getopts option processing and -b won't be considered.

The other alternative would be to use some delimiter like , and parse it yourself

./script -d /dev,/home/work -b /backup

Answered By - Diego Torres Milano

Answer Checked By - Marilyn (PHPFixing Volunteer)

Read More

[FIXED] How to compare filenames in two text files on Linux bash?

Issue

I have two lists list1 and list2 with a filename on each line. I want a result with all filenames that are only in list2 and not in list1, regardless of specific file extensions (but not all). Using Linux bash, any commands that do not require any extra installations. In the example lists, I do know all file extensions that I wish to ignore. I made an attempt but it does not work at all, I don't know how to fix it. Apologies for my inexperience.

I wish to ignore the following extensions: .x .xy .yx .y .jpg

list1.txt

text.x
example.xy
file.yx
data.y
edit
edit.jpg

list2.txt

text
rainbow.z
file
data.y
sunshine
edit.test.jpg
edit.random

result.txt

rainbow.z
sunshine
edit.test.jpg
edit.random

My try:

while read LINE
    do
    line2=$LINE
    sed -i 's/\.x$//g' $LINE $line2
    sed -i 's/\.xy$//g' $LINE $line2
    sed -i 's/\.yx$//g' $LINE $line2
    sed -i 's/\.y$//g' $LINE $line2 
    then sed -i -e '$line' result.txt;
    fi
done < list2.txt

Edit: I forgot two requirements. The filenames can have . in them and not all filenames must have an extension. I know the extensions that must be ignored. I ammended the lists accordingly.

Solution

An awk solution might be more efficient for this task:

awk '
              { f=$0; sub(/\.(xy?|yx?|jpg)$/,"",f) }
    NR==FNR   { a[f]; next }
    !(f in a)
' list1.txt list2.txt > result.txt

Answered By - M. Nejat Aydin

Answer Checked By - Cary Denson (PHPFixing Admin)

Read More

[FIXED] How to run a PowerShell script without displaying a window?

Issue

How is it possible to run a PowerShell script without displaying a window or any other sign to the user?

In other words, the script should run quietly in the background without any sign to the user.

Extra credit for an answer that does not use third party components :)

Solution

You can either run it like this (but this shows a window for a while):

PowerShell.exe -WindowStyle hidden { your script.. }

Or you use a helper file I created to avoid the window called PsRun.exe that does exactly that. You can download the source and exe file from Run scheduled tasks with WinForm GUI in PowerShell. I use it for scheduled tasks.

Edited: as Marco noted this -WindowStyle parameter is available only for V2 and above.

Answered By - stej

Answer Checked By - Terry (PHPFixing Volunteer)

Read More

[FIXED] How do I extract the string that follows another string from multiple files?

Issue

I have a lot of files in the folder filesToCheck, some examples given below. I need the output result.txt as also shown below. I think the regex CAKE_FROSTING\(\".*\" is needed somehow for this task, but I am not well versed in bash scripting. I can use linux bash with any commands that do not require extra installations.

file1.cpp

something
CAKE_FROSTING("is.simply.the.best", "[no][matter][what]") { DO(something(0) == 1); }

file2.h

something else
CAKE_FROSTING(
"is.kinda.neat", 
"[i][agree]") something else
something more

file3.cpp

hello

file4.cpp

random_text CAKE_FROSTING("Can be nice") "more random text"

CAKE_CREAM("totally.sucks", "[trust][me]")

fileEmpty.h

---

result.txt

is.simply.the.best
is.kinda.neat
Can be nice

Edit: I tried

awk '"CAKE_FROSTING\("{print $2}' filesToCheck/file1.cpp

but this gives the wrong output "[no][matter][what]") and only runs on one file.

Solution

Using GNU sed

$ sed -Enz 's/.*CAKE_FROSTING\(\n?"([^"]*).*/\1\n/p' /path/to/filesToCheck/*
is.simply.the.best
is.kinda.neat
Can be nice

Answered By - HatLess

Answer Checked By - Cary Denson (PHPFixing Admin)

Read More

[FIXED] How do I extract the string that follows another string from multiple files?

Issue

I have a lot of files in the folder filesToCheck, some examples given below. I need the output result.txt as also shown below. I think the regex CAKE_FROSTING\(\".*\" is needed somehow for this task, but I am not well versed in bash scripting. I can use linux bash with any commands that do not require extra installations.

file1.cpp

something
CAKE_FROSTING("is.simply.the.best", "[no][matter][what]") { DO(something(0) == 1); }

file2.h

something else
CAKE_FROSTING(
"is.kinda.neat", 
"[i][agree]") something else
something more

file3.cpp

hello

file4.cpp

random_text CAKE_FROSTING("Can be nice") "more random text"

CAKE_CREAM("totally.sucks", "[trust][me]")

fileEmpty.h

---

result.txt

is.simply.the.best
is.kinda.neat
Can be nice

Edit: I tried

awk '"CAKE_FROSTING\("{print $2}' filesToCheck/file1.cpp

but this gives the wrong output "[no][matter][what]") and only runs on one file.

Solution

Using GNU sed

$ sed -Enz 's/.*CAKE_FROSTING\(\n?"([^"]*).*/\1\n/p' /path/to/filesToCheck/*
is.simply.the.best
is.kinda.neat
Can be nice

Answered By - HatLess

Answer Checked By - Clifford M. (PHPFixing Volunteer)

Read More

[FIXED] How do I extract specific strings from multiple files and write them to .txt in bash?

Issue

I have a lot of files in the folder filesToCheck, some examples given below. I need the output result.txt as also shown below. I can use linux bash with any commands that do not require extra installations.

The try-out below (with help from stackoverflow) has two problems when I execute it. It only looks for one instance of CAKE_FROSTING despite the global flag g and the file result.txt remains empty despite > result.txt.

sed -Enz 's/.*CAKE_FROSTING\(\n?"([^"]*).*/\1\n/gp' filesToCheck/* > result.txt

What do I need to change?

---

file1.cpp

something
CAKE_FROSTING("is.simply.the.best", "[no][matter][what]") { DO(something(0) == 1); }

file2.h

something else
CAKE_FROSTING(
"is.kinda.neat", 
"[i][agree]") something else
something more

file3.cpp

random_text CAKE_FROSTING("Can be nice") "more random text"

CAKE_CREAM("totally.sucks", "[trust][me]")
random_text CAKE_FROSTING("Can be very nice") "indeed"

---

desiredResult.txt

is.simply.the.best
is.kinda.neat
Can be nice
Can be very nice

currentResult command line output:

is.simply.the.best
is.kinda.neat
Can be nice

Solution

Assuming the string CAKE_FROSTING occurs once per line, you can try this sed

$ sed -En ':a;N;s/.*CAKE_FROSTING\(\n?"([^"]*).*/\1/p;ba' filesToCheck/*
is.simply.the.best
is.kinda.neat
Can be nice
Can be very nice

Answered By - HatLess

Answer Checked By - David Marino (PHPFixing Volunteer)

Read More

[FIXED] Why am I getting unexpected-token error in this script?

Issue

I want to loop through all the files in the directory and check their perms (if user wants) but for some reason I'm getting this error:

./perms.sh: line 12: syntax error near unexpected token `;;'
./perms.sh: line 12: `  r) if [ -r $i ] then echo 'True' else echo 'False' fi ;;'

here is my code:

#!/bin/bash

for i in *
do
    echo "Do you want to check rights for $i (y/n)"
    read marzi
    if [ $marzi = 'y' ]
then
    echo 'which commands to check? '
    read check
    case $check in
    r) if [ -r $i ] then echo 'True' else echo 'False' fi ;;
    w) if [ -w $i ] then echo 'True' else echo 'False' fi ;;
    x) if [ -x $i ] then echo 'True' else echo 'False' fi ;;
    *) echo  'unrecognized!' ;;
esac
else
    echo "skipped $i"
fi
done

does this have something to do with apostrophe?

Solution

maybe correct inline if format should be

if [ -r $i ]; then echo 'True'; else echo 'False'; fi

make sure to make change for r and w and x

Answered By - stuffy

Answer Checked By - Marilyn (PHPFixing Volunteer)

Read More

[FIXED] How to merge or append exported info of two different Exchange Powershell scripts?

Issue

I'm having a problem that I haven't been abble to resolve, I need to merge two scripts or append the second script exported info into the same delimited CSV file (created by the first script).

This is the first script:

Get-Mailbox -ResultSize Unlimited | Select-Object AddressBookPolicy, ProhibitSendQuota, SamAccountName, UserPrincipalName, WhenMailboxCreated, Alias, OrganizationalUnit, CustomAttribute1, DisplayName, PrimarySmtpAddress, RecipientType, RecipientTypeDetails, WindowsEmailAddress, WhenChanged, WhenCreated | export-csv -NoTypeInformation .\Mailboxes_filtered.csv -Delimiter ";" -Encoding unicode

And this the second one:

Get-Mailbox -ResultSize Unlimited | Get-MailboxStatistics | Select DisplayName, StorageLimitStatus, TotalItemSize | export-csv -NoTypeInformation .\Mailboxes_filtered.csv -Delimiter ";" -Encoding unicode

PS: I'm using Exchange 2010.

I managed to get some success using "AddContent -Path .\Mailboxes_filtered.csv", but the added info appeared under the delimited cells on the CSV file instead of showing up beside and organized in the same way, I guess it happened because in this case the -Delimited ";" parameter is not accepted...

Those two scripts work, I just need to merge or append the exported info into the same CSV file.

Solution

You don't need to combine the two csv, you need to add the additional properties to each record. Since the record already has a DisplayName property, you would either need to overwrite it or change the property name to something else.

You can use calculated properties to tack on the additional properties. The flow will be take each mailbox one by one and look up the stats for that mailbox. Grab the properties you want out of the stats and add with calculated properties.

$mailboxlist = Get-Mailbox -ResultSize Unlimited | Select-Object AddressBookPolicy, ProhibitSendQuota, SamAccountName, UserPrincipalName, WhenMailboxCreated, Alias, OrganizationalUnit, CustomAttribute1, DisplayName, PrimarySmtpAddress, RecipientType, RecipientTypeDetails, WindowsEmailAddress, WhenChanged, WhenCreated 

$results = foreach($mailbox in $mailboxlist){
    $stats = Get-MailboxStatistics -Identity $mailbox.UserPrincipalName | Select DisplayName, StorageLimitStatus, TotalItemSize

    $mailbox | Select-Object *, @{n='StorageLimitStatus';e={$stats.StorageLimitStatus}}, @{n='TotalItemSize';e={$stats.TotalItemSize}}
}

$results | Export-Csv -NoTypeInformation .\Mailboxes_filtered.csv -Delimiter ";" -Encoding unicode

Answered By - Doug Maurer

Answer Checked By - Senaida (PHPFixing Volunteer)

Read More

[FIXED] How to Forloop Through A Chocolatey Package Install with something Similar to a requirements.txt

Issue

So I am using Chocolatey to help with Package installs on a computer and am wondering if there is a way to for loop through this in powershell. I am more familiar with for loops in Bash with Linux and Mac then I am with Windows, so was wondering if someone can give me some insights,

I basically am running the following scenario:

choc install <package_name> -y

And I need to install all of these packages at once:

Chocolatey v0.10.15
androidstudio 3.5.3.0
atom 1.41.0.20191029
awk 4.2.132
chocolatey 0.10.15
chocolatey-core.extension 1.3.5
chocolatey-dotnetfx.extension 1.0.1
chocolatey-visualstudio.extension 1.8.1
chocolatey-windowsupdate.extension 1.0.4
CrystalReports2010Runtime 13.0.18
curl 7.67.0
docker 99.0.0
docker-cli 19.03.3
DotNet3.5 3.5.20160716
dotnetcore-runtime 3.1.0
dotnetcoresdk 1.0.1
dotnetfx 4.8.0.20190930
filezilla 3.46.0
flashplayeractivex 32.0.0.303
flashplayerplugin 32.0.0.293
gawk 5.0.1
gimp 2.10.14
git 2.24.0
git.install 2.24.0
gitlab-runner 12.5.0
GoogleChrome 80.0.3987.116
gpg4win-vanilla 2.3.4.20191021
grafana 6.3.6
grep 2.1032
jdk8 8.0.231
jenkins-x 2.0.1082
jq 1.6
KB2533623 1.0.4
KB2919355 1.0.20160915
KB2919442 1.0.20160915
KB2999226 1.0.20181019
KB3033929 1.0.5
KB3035131 1.0.3
kubernetes-cli 1.17.0
less 5.51.0.20191024
minikube 1.6.1
mysql-odbc 5.3.12
nano 2.5.3
nodejs.install 13.3.0
openssh 8.0.0.1
openvpn 2.4.7
postgresql 12.1
postgresql12 12.1
postman 7.13.0
prometheus 2.2.1
prometheus-wmi-exporter.install 0.9.0
putty 0.73
putty.portable 0.73
python 3.8.0
python3 3.8.0
random-number-generator 3.0.0
ruby 2.6.5.1
rubymine 2019.2.4
runinbash 0.1.1
sed 4.7
teamviewer 15.0.8397
unzip 6.0
vagrant 2.2.6
vault 1.3.0
vcredist140 14.23.27820
vcredist2013 12.0.40660.20180427
vcredist2015 14.0.24215.20170201
vim 8.1.2318
visioviewer 16.0
visioviewer2016 16.0
visualstudio-installer 2.0.1
visualstudio2019community 16.3.10.0
vlc 3.0.8
Wget 1.20.3.20190531

How can I accomplish this?

Solution

When you run an external command (or any command), the output of that command can be piped into a PowerShell command. Since your command outputs single lines containing a package name, that output can be piped into Foreach-Object where each package (object in PowerShell terms) can be processed.

<package output command> | Foreach-Object { choc install $_ -y }

The current object being processed by Foreach-Object is $_ or $PSItem.

Answered By - AdminOfThings

Answer Checked By - Cary Denson (PHPFixing Admin)

Read More

[FIXED] How to urlencode data for curl command?

Issue

I am trying to write a bash script for testing that takes a parameter and sends it through curl to web site. I need to url encode the value to make sure that special characters are processed properly. What is the best way to do this?

Here is my basic script so far:

#!/bin/bash
host=${1:?'bad host'}
value=$2
shift
shift
curl -v -d "param=${value}" http://${host}/somepath $@

Solution

Use curl --data-urlencode; from man curl:

This posts data, similar to the other --data options with the exception that this performs URL-encoding. To be CGI-compliant, the <data> part should begin with a name followed by a separator and a content specification.

Example usage:

curl \
    --data-urlencode "paramName=value" \
    --data-urlencode "secondParam=value" \
    http://example.com

See the man page for more info.

This requires curl 7.18.0 or newer (released January 2008). Use curl -V to check which version you have.

You can as well encode the query string:

curl --get \
    --data-urlencode "p1=value 1" \
    --data-urlencode "p2=value 2" \
    http://example.com
    # http://example.com?p1=value%201&p2=value%202

Answered By - Jacob Rask

Answer Checked By - Katrina (PHPFixing Volunteer)

Read More

[FIXED] How can I declare and use Boolean variables in a shell script?

Issue

I tried to declare a Boolean variable in a shell script using the following syntax:

variable=$false

variable=$true

Is this correct? Also, if I wanted to update that variable would I use the same syntax? Finally, is the following syntax for using Boolean variables as expressions correct?

if [ $variable ]

if [ !$variable ]

Solution

Revised Answer (Feb 12, 2014)

the_world_is_flat=true
# ...do something interesting...
if [ "$the_world_is_flat" = true ] ; then
    echo 'Be careful not to fall off!'
fi

---

Original Answer

Caveats: https://stackoverflow.com/a/21210966/89391

the_world_is_flat=true
# ...do something interesting...
if $the_world_is_flat ; then
    echo 'Be careful not to fall off!'
fi

From: Using boolean variables in Bash

The reason the original answer is included here is because the comments before the revision on Feb 12, 2014 pertain only to the original answer, and many of the comments are wrong when associated with the revised answer. For example, Dennis Williamson's comment about bash builtin true on Jun 2, 2010 only applies to the original answer, not the revised.

Answered By - miku

Answer Checked By - Clifford M. (PHPFixing Volunteer)

Read More

[FIXED] How do I split a string on a delimiter in Bash?

Issue

I have this string stored in a variable:

IN="bla@some.com;john@home.com"

Now I would like to split the strings by ; delimiter so that I have:

ADDR1="bla@some.com"
ADDR2="john@home.com"

I don't necessarily need the ADDR1 and ADDR2 variables. If they are elements of an array that's even better.

---

After suggestions from the answers below, I ended up with the following which is what I was after:

#!/usr/bin/env bash

IN="bla@some.com;john@home.com"

mails=$(echo $IN | tr ";" "\n")

for addr in $mails
do
    echo "> [$addr]"
done

Output:

> [bla@some.com]
> [john@home.com]

There was a solution involving setting Internal_field_separator (IFS) to ;. I am not sure what happened with that answer, how do you reset IFS back to default?

RE: IFS solution, I tried this and it works, I keep the old IFS and then restore it:

IN="bla@some.com;john@home.com"

OIFS=$IFS
IFS=';'
mails2=$IN
for x in $mails2
do
    echo "> [$x]"
done

IFS=$OIFS

BTW, when I tried

mails2=($IN)

I only got the first string when printing it in loop, without brackets around $IN it works.

Solution

You can set the internal field separator (IFS) variable, and then let it parse into an array. When this happens in a command, then the assignment to IFS only takes place to that single command's environment (to read ). It then parses the input according to the IFS variable value into an array, which we can then iterate over.

This example will parse one line of items separated by ;, pushing it into an array:

IFS=';' read -ra ADDR <<< "$IN"
for i in "${ADDR[@]}"; do
  # process "$i"
done

This other example is for processing the whole content of $IN, each time one line of input separated by ;:

while IFS=';' read -ra ADDR; do
  for i in "${ADDR[@]}"; do
    # process "$i"
  done
done <<< "$IN"

Answered By - Johannes Schaub - litb

Answer Checked By - Candace Johnson (PHPFixing Volunteer)

Read More