Office 365 Windows apps not upgrading to latest

 If you have ever come across the situation when your colleagues have features or annoyances in Office 365 desktop applications that your Outlook / Excel does not have?
And no matter how many times you manually initiate Office Update under File / (Office) Account / Update Options / Update Now nothing new comes?

We do a lot of collaborative editing of tracking lists in Excel where private, temporary Sheet Views of Data Filtering and Sorting come very handy so that one does not alter the filter criteria for the other editing the same list.

I did not have this option since I could not upgrade further than the (20)20-02(Feb) version of the apps:

You can also see the culprit: after much research, uninstall-reinstall that did not help I figured that the Semi-Annual Enterprise Channel is not delivering the updates (yet).

If you do not want to wait you have to change the Office Update Channel: https://docs.microsoft.com/en-us/deployoffice/change-update-channels however the enterprise methods linked in this article did not work for me.

Thus I have researched some more and changed my update channel manually through changing my registry: https://www.solver.com/switching-office-365-monthly-update-channel however the changes described in this article also were not enough for me.

So what I did:

1. start regedit
2. navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\ClickToRun\Configuration
3. Change CDNBaseURL to http://officecdn.microsoft.com/pr/492350f6-3a01-4f97-b9c0-c7c6ddf67d60
4. Change AudienceId to 492350f6-3a01-4f97-b9c0-c7c6ddf67d60
5. Change UpdateChannel to http://officecdn.microsoft.com/pr/492350f6-3a01-4f97-b9c0-c7c6ddf67d60
6. Change UpdateChannelChanged to True
7. Start Excel (or any other Office application)
8. Initiate File / (Office) Account / Update Options / Update Now
9. Exit Excel and wait for the updates to download and install.
10. Open Excel: File / Account now I am on version (20)20-10(Oct):

Same with Word:

Office Update Channel changed to Current!

I also did some restarts in between, #6 and #7 above. You should do it too, if the above steps alone do not work for you straight away.

Reccursive directory list to CSV in a single

 How to create a recursive list of all files and separate size, path, filename and extension into CSV fields in a single command line of standard Windows 10 CMD?

TL;DR for the current directory, not including directories:

for /f "delims=*" %A in ('dir /s /b /a:-d') ^
do @echo %~zA,"%~pA","%~nA",%~xA >>dirlist.csv

(feel free to omit the carrot and put it on a single line)

• Details: you can use the Parameter Expansion known from parsing the command line arguments on parameters of the FOR command (yes, even if you indicate %L you can have %M, %N and so on as well) https://ss64.com/nt/syntax-args.html and
• https://ss64.com/nt/for.html
  

Have fun!

Configuring wifi-ap and hostapd on Ubuntu 20.04

If you have a fresh new, shiny Ubuntu focal installed and you are thinking to set up wifi sharing (an Access Point) in the part of your apartment that computer is located in installing and starting the wifi-ap snap is a great way to do so.
Especially if you, like me have installed it without a graphical desktop, so setting up an AP using the Network Manger GUI is now straightforward.

Make sure you have snap:

sudo apt install snapd

Then install wifi-ap:

snap install wifi-ap

Theoretically an automated setup will make it work straight after install, you can check that with:

wifi-ap.status

This did not work on my box, because I do not have a wlan0 interface, instead I had to configure it with:
wifi-ap.config set wifi.interface=wlp1s0

Configuring is very easy as far as you can live with the few options it gives you:
root@XXXXX:~# wifi-ap.config get
debug: false
dhcp.lease-time: 12h
dhcp.range-start: 10.0.60.2
dhcp.range-stop: 10.0.60.199
disabled: false
share.disabled: false
share.network-interface: enp3s0
wifi.address: 10.0.60.1
wifi.channel: 1
wifi.country-code: SG
wifi.hostapd-driver: nl80211
wifi.interface: wlp1s0
wifi.interface-mode: direct
wifi.netmask: 255.255.255.0
wifi.operation-mode: g
wifi.security: wpa2
wifi.security-passphrase: XXXXXXXXXX
wifi.ssid: XXXXXXXXXX

You can turn off the snap and back on with:
wifi-ap.config set disabled=true
wifi-ap.config set disabled=false

And restart with:
wifi-ap.status restart-ap

After these my hostapd was running but the few settings above did not allow me to set the settings to the optimal speed with the tuning options in hostapd.conf I have already experimented with earler:
ht_capab=[HT40+][SHORT-GI-20][SHORT-GI-40][DSSS_CCK-40][TX-STBC][RX-STBC1]

And editing /var/snap/wifi-ap/355/hostapd.conf does not help as the snap regenerates it upon every startup in /snap/wifi-ap/355/bin/ap.sh which is a read-only mounted squashfs:
root@XXXXXX:~# mount |grep snap
/var/lib/snapd/snaps/wireless-tools_7.snap on /snap/wireless-tools/7 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core_9665.snap on /snap/core/9665 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/wifi-ap_355.snap on /snap/wifi-ap/355 type squashfs (ro,nodev,relatime,x-gdu.hide)

So I have turned to this page on how to unmount and unpack the squashfs, modify the generating script https://askubuntu.com/questions/1046606/cannot-edit-system-file-snap-phpstorm-even-with-root-account-in-ubuntu-18-04/1046677:

cd ~

sudo umount /snap/wifi-ap/355
sudo unsquashfs /var/lib/snapd/snaps/wifi-ap_355.snap

This is when you modify anything you want in the squashfs-root directory except I could not unmount because the service process was still using the mount point. I figured out the PID for it with:
fuser -m /snap/wifi-ap/355
And killed it with:
kill -9

Then added the above config line to the config generation part of the script just before the wmm_* section, recreated the squasfs, backed up the old snap image and placed the newly generated there:

sudo mksquashfs squashfs-root wifi-ap_355.snap

sudo mv /var/lib/snapd/snaps/wifi-ap_355.snap /var/lib/snapd/snaps/wifi-ap_355.snap.backup

mv ~/wifi-ap_355.snap /var/lib/snapd/snaps/wifi-ap_355.snap

Not very elegant but then I just restarted my computer instead of figuring out how to restart the service I have killed.

I had everything in order, hostapd running in the wifi-ap snap with the updates options.

So long, and thanks for all the fish!

Image resolution for OCR and how to check it

The resolution of the documents consisting of scanned pages determines how much information they store about the visual content, so how well or easily those pages are readable. This is usually measured in DPI (Dot Per Inch - how many pixels are covering a 2.54cm part of the page) so that it is independent of the page size. Horizontal and vertical DPI is usually the same, and for normal documents 300 DPI is optimal (higher is only needed for documents with small print). The minimum acceptable DPI for production OCR processing is 200 DPI (as advised by vendors of several OCR engines).
For the usual business documents of A4 and Letter standards the below table shows the amount of information contained and describing the visual content of the complete page at different usual DPIs (as well as horizontal and vertical size in inches, millimeters, pixels and DPI):

It can be seen that a 200 DPI image has less than half the information about the page (practically a 3.74 megapixel photo - worse than what any smartphone can take in 2020) compared to the optimal 300 DPI (approx. 8.5 megapixels). Anything below has even more scarce information about the page: 150 DPI only has one quarter, 100 DPI only has one tenth of the information.

Consequently a modern OCR solution using traditional engines (ABBYY, Nuance, Tesseract, RecoStart whichever company has them acquired at the moment) can be expected to perform adequately if feed with 200-300 DPI documents.

Up scaling scanned images just inserts rows and columns of smartly averaged pixels between the original pixels but this does not add the missing information: what was really there originally on the document. This is automatically performed by the engines anyways, but is not expected to make the results better, just enables processing low resolution documents (still usually producing substandard results).

Effect of JPEG: lossy compression 

JPEG uses lossy compression, this simply means, that even though you input the specific pixel information you have scanned (what color is each location in the page with a specific resolution), it does not store this information as given: it encodes 8x8 blocks converted into frequencies with a quality setting. If that is high, the JPEG closely resembles the image, so it is almost as good for OCR as the original and is larger. Lower quality adds noise and makes OCR results worse.

On comparison TIFF G4, PNG etc. formats store pixel as received, there is no degradation due to compression, this is called loss-less compressions and usually results in larger files.

How to figure out the resolution of your input files

For JPEG, PNG, TIFF etc. open them in e.g. IrfanView https://www.irfanview.com/ and fin.d the resolution in the bottom left in the status bar or press “I” for “I”mage properties. In case of TIFFs switch to the page you one to see the resolution for, they do not necessarily have the same resolution. Resolution may slightly differ from the numbers in the above table.

Finding the resolution of the images in PDFs

You can use the pdfimages command from popplers utils (free, opensource software).

Using WSL on Windows 10

Using msys2 on other Windows versions

Please note, PDFimages may calculate DPI wrong (for example if images are stored rotated) but the resolutions you can always compare to the table above.

How to install pdfimages

Using Windows Subsystem for Linux on Windows 10

1. Enable Windows Subsystem for Linux
2. Install e.g. Ubuntu from the Microsoft Store
3. Start bash
4. Install the poppler-util package (and its dependencies)

$ sudo apt install -y poppler-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
poppler-utils is already the newest version (0.62.0-2ubuntu2.10).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Using msys2

https://www.msys2.org/ is a free and open-source platform for multiple Windows versions (excluding Windows XP and below)

After installing msys2, install the package (the example below is for 64 bit installations):

c:\msys64\usr\bin\bash -lic “pacman -S mingw-w64-x86_64-poppler“

Troubleshooting Wireguard VPN on Windows 10, Android and Linux

I have had my share of pain over the compexity / slowness / incompatibilities / vulnerabilities of using Cisco,
To me it seems the primary problem with Wireguard are twofold:

1. Not having enough experience in the community (blog posts, walk-throughs, how-tos etc.) to set up all kinds of arrangements besides the usual site-to-site and cloud VPN jump-host.
2. Clients offer less the adequate error messages that could help with debugging / troubleshooting.
3. Clients across platforms are not consistent.

I am writing this for two reasons:

• helping fellow users with similar situations
• and to give feedback to the developers (will try to figure there to submit reports and which of the issues are known already).

Things to fix / disambiguate / document in the various WireGuard components:

1. The Android client does not have the nice log viewer that is part of the Windows client - and that helped me to see what is (not) happening) 
2. You can export the  log form the Android client is full of UI related Java messages, unlike the clean log of the Windows client - it really makes it very hard to comprehend what is going on.
3. The Android client just disappears after a while (even with the PersistentKeepalive set to 25), so suddenly the VPN protection disappears without any notification. This did not happen to the OpenVPN Android client, so probably just have to tell Android not to evict / suspend the VPN software somehow.
4. The error message "bad address" (Android client, creating configuration from scratch) is misleading or not informative enough: got it for example for 192.168.1.1/24 (should be /32 or 192.168.1.0/24) - could correct it automatically or at least be more informative telling you what is wrong.
5. It is hard to figure where Wireguard is logging on linux with systemd.
   Is it logging at all?
   - Could not find any trace of the failed connection attempts, so it was really hard to tell, if my DNS, my port forwarding or my Wireguard config is wrong (was the latter).
   - Could not find messages about 192.168.1.2/24 being inaccessible (overridden) if there is a 192.168.1.3/24 peer afterwards, so have to use /32 peers even if the server interface address is communicating on a 192.168.1.1/24 address with both of the clients.
   - systemd startup log did not habve any relevant messages either

What my mistakes and symptoms were:

1. Accidentally mixed up a private and a public key. Wiregoard just silently fails, does not tell you that there was a connection attempt but the key was wrong. Could have been any network related inaccessibility as well...
2. Did not know how to configure the peer addresses each for /32 so that they don't interfere but both can communicate with the /24 server interface.

USB3 disks cannot provide decent performance

Everyone kind of knows and suggest this, but Now I have empirical evidence: I have switched the connection of seven SATA HDDs in two enclosures to my PC

• from two direct USB3 connections
• to two eSATA connections of an Akitio (single) ThunderBolt(1)-(dual)eSATA bridge (connected to an Apple ThunderBolt 3 to ThunderBolt 2 converter)

And even in this complicated TB3-TB2-TB1-2xeSATA-7xSATA performs waaaay better than the old 2xUSB3-7xSATA setup:

Screenshot shows 692megbit/sec total active torrent speed that does not even include torrent protocol and TCP/IP overhead and any other traffic on the host. That actually goes though my (gigabit Ethernet) router too. Seems gigabit fiber internet connections are not just a promise.

Before switching the disk access path it never went above 300megabit/sec.

Probable reason is USB3 does not implement DMA (compatible USB3.1 device-host pairs would do) while TB1-2-3 are basically PCIe connections channeled outside the computer case. :)

VMware slow because of MsMpEng.exe (Microsoft AntiMalware Service Executable)

Have you noticed how much CPU MsMpEng.exe (AntiMalware Service Executable) uses while running vmware? (for me 10-25% of the CPU if the VM is active).

I tried to add exceptions for Windows Defender in Windows Defender Security Center for:

• program vmware-vmx.exe
• and file types vmdk, nvram, vmem and vmx.

Seems to help a lot... now it stays below 2% CPU all the time :)

Agressively reducing WinSXS with DISM

In my recent quest to reduce virtual server images I noticed that a huge part of the total image is taken up by the WinSXS folder. Here is a comprehensive list of DISM commands you can run to reduce that beyond what is possible by running "Disk Cleanup" from the properties of the C: drive.

Always use a command prompt that is run as Administrator (start menu, type CMD, right click on cmd.exe, Run as Administrator)!

for Windows 8.x, 10 and Windows Server 2012 (R2)

It's always a good idea and can avoid a lot of strange error messages to make sure the Deployment Image is in a healthy condition:
Dism /Online /Cleanup-Image /RestoreHealth

Use this to understand how much of your WinSXS is actually shared with (actively used by) Windows, and how much is taken up by unused features / packages and other stuff:
dism.exe /Online /Cleanup-Image /AnalyzeComponentStore

Use this to get rid of the files from before a service pack was applied:
Dism /online /Cleanup-Image /SPSuperseded

And this one will make sure only the latest necessary version of each file in WinSXS is kept around:
Dism /online /Cleanup-Image /StartComponentCleanup /ResetBase

More aggressive cleansing

Should you want to get rid of all the unused features as well, you can do this:
DISM.exe /Online /English /Get-Features /Format:Table > features.lst

Then you can use this short bash command line to create a .bat file to remove all files that belong to disabled features: (use your Linux, or for windows: use the Linux subsystem in Windows 10, Cygwin on earlier versions of Windows to run bash)
grep '| Disabled\s*$' CMOD-features.lst|cut -d '|' -f 1|while read; do echo "DISM.exe /Online /Disable-Feature /featurename:${REPLY// /} /Remove"; done > remove-disabled-features.bat

Then just run the created .bat file to remove all the rest of the cruft. In theory Windows will be able to reinstall these files if you turn on any of the features, for which the files were removed, but I would not go ahead with the removal until I am reasonable sure I only remove what I will not need. YMMV!

For Windows 7 and Windows Server 2008 R2

It's always a good idea and can avoid a lot of strange error messages to make sure the Deployment Image is in a healthy condition:
dism /online /Cleanup-Image /scanhealth

Use this to get rid of the files which were deprecated by installed Service Packs:
DISM.exe /Image:C:\test\offline /Cleanup-Image /spsuperseded /hidesp

You can get the list of packages, and seems safe to remove the ones that are not "Installed" but "Staged" or "Superseded"

dism /online /Format:Table /Get-Packages > packages.lst

Then remove them, for example:
dism /online /remove-package /packagename:Microsoft-Windows-CodecPack-Basic-Package~31bf3856ad364e35~amd64~~6.1.7601.17514

You can also list the installed features both enabled and disabled:
dism /online /get-features >features.lst

But so far I have found no way on the older Windows versions to remove the files from WinSXS that belong to disabled features...

Extract all your Pictures, Videos, PDFs from iPhone backup!

I hate the fact that is so hard to retrieve all my files that I have on the iPhone (this is one of the reasons why I am switching from iPhone to an android device). They are all in the iPhone backup, but with their names scrambled, without extensions is impossible to sort thousands of files.

But not any longer, see how you can get access to all your files and data!

1. Create a backup of your iPhone on your computer, that is _not_ encrypted (as described here)
2. Find the files of your backup:
   
   • on Windows 8.x and 10 it is here: c:\Users\\AppData\Roaming\Apple Computer\MobileSync\Backup\
   • on Windows 7 and Vista: C:\Users\\AppData\Roaming\Apple Computer\MobileSync\Backup\
   • on Windows XP: C:\Documents and Settings\\Application Data\Apple Computer\MobileSync\Backup 
   • on OS X: ~/Library/Application Support/MobileSync/Backup/
3. Each backup name is a 40 character long SHA (seems like garbage: 00d07a612db092c28c316244cce7d9199f23da33).
   Inside of them you will find another 256 folders named 00-ff, and inside those are your files, also with the long hash (garbage) names.
4. I have created the below script, that will:
   - sort your files into target folders by type, and add their correct extensions
   - decode the Apple Binary Property Lists and SQLite databases, so that you can view and search them in clear text! (databases will also be copied there besides the dump if you want to open and query them!)
5. You will need bash, sqlite3 and plistutil to run it, either on your Linux, or for windows: use the Linux subsystem in Windows 10, Cygwin on earlier versions of Windows.
6. Please do not forget to replace the backup folder name and the target directory name with yours!

#!/bin/bash
#
# Created to compare speed and compression on a VM image
# to be run in the VM working directory e.g. /mnt/d/VMs
#

# Find the utils we will need
if ! which sqlite3 >/dev/null; then echo "Please install sqlite3 to dump database file contents!"; exit; fi
if ! which plistutil >/dev/null; then echo "Please install sqlite3 to dump database file contents!"; exit; fi

# Identify file types
# Put target directory here!
TDIR="/mnt/c/Users//Desktop/iPhone6.HU"
mkdir -p "$TDIR/Pictures/JPEGs/" "$TDIR/Pictures/PNGs/" "$TDIR/Text/" "$TDIR/XML/" "$TDIR/PDF/" "$TDIR/PlistToXML/" "$TDIR/Database/" "$TDIR/Audio/" "$TDIR/Contacts/" "$TDIR/Pictures/TIFFs/" "$TDIR/Fonts/" "$TDIR/Others/" "$TDIR/Movies/"
 
# Put your backup folder here!
file --no-pad /mnt/c/Users//AppData/Roaming/Apple\ Computer/MobileSync/Backup//*/* |

while read
do
  SRC="${REPLY%%:*}"
  FLNAME="${SRC##*/}"
  case "${REPLY#*: }" in
  (JPEG*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Pictures/JPEGs/$FLNAME.jpg"
    ;;
  (PNG*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Pictures/PNGs/$FLNAME.png"
    ;;
  (ASCII\ text*|*Unicode\ text*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Text/$FLNAME.txt"
    ;;
  (XML\ document\ text*)
    cp --preserve --no-clobber "$SRC" "$TDIR/XML/$FLNAME.xml"
    ;;
  (PDF*)
    cp --preserve --no-clobber "$SRC" "$TDIR/PDF/$FLNAME.pdf"
    ;;
  (Apple\ binary\ property*)
    plistutil -i "$SRC" -o "$TDIR/PlistToXML/$FLNAME.xml"
    ;;
  (SQLite\ 3*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Database/$FLNAME.sqlite"
    sqlite3 "$SRC" .dump >"$TDIR/Database/$FLNAME.sql"
    ;;
  (MPEG-4\ LOAS*|*AAC*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Audio/$FLNAME.aac"
    ;;
  (vCard*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Contacts/$FLNAME.vcard"
    ;;
  (TIFF*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Pictures/TIFFs/$FLNAME.tiff"
    ;;
  (TrueType*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Fonts/$FLNAME.ttf"
    ;;
  (*QuickTime\ movie*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Movies/$FLNAME.mov"
    ;;
  (*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Others/$FLNAME"
  esac
done
 
Sample output folder:

Enjoy!

Should you add more interesting file types, please add them in the comments section!