Example GitHub Action workflow to auto push a specified directory to a different GitHub repo

I made an example GitHub Action workflow that will automatically push all files in a specified directory to a different GitHub repo using a GitHub deploy key. By enabling GitHub Pages for the destination repo, you can serve content dynamically generated by a GitHub action workflow/job.

Check it out!

Use cases

  • Publish static (gatsby, jekyll etc.) site to separate GitHub repository
  • Publish build results to a web endpoint
  • Publish istanbul test coverage from unit/integration tests
  • Publish a custom badge like shields.io (e.g a test coverage badge)

Why?

  • Free serving of public websites with git history (public repositories don’t cost anything)
  • Don’t have to pay for services like codecov.io or coveralls.io
  • Don’t bloat your main repo with a long history of ever changing files.
  • Don’t risk exposing any personal github token from a github action (a leak which which could give someone access to your whole GitHub account).

Hvordan Vippse til kontonummer

Hvordan Vippse penger til et kontonummer i stedet for et telefonnummer:

How to send money using the Vipps apps to an arbitrary bank account number instead of a phone number:

1 - Tap Scan

2 - Choose Bill
3 - Take a picture of anything (not a bill)

4 - Choose Type in

5 - Manually enter the account number, message and amount and send the money

6 - Now you know how 😎

Sniffing SSL traffic on Android (Vipps Payment App)

I previously attempted to do the same with an early version of the Vipps (Norwegian easy payment app), but the app has changed a lot since then. I decided to try again with the current version, and I learned that it has implemented quite a few new countermeasures preventing the old method from working.

The process

First download Vipps app on an Android phone, then connect it and copy the app over USB:

adb shell pm list packages | grep vipps
adb shell pm path no.dnb.vipps

adb pull /data/app/no.dnb.vipps-QLoJ4giHA8cXj7Pl6RI20A==/base.apk
adb pull /data/app/no.dnb.vipps-QLoJ4giHA8cXj7Pl6RI20A==/split_config.arm64_v8a.apk
adb pull /data/app/no.dnb.vipps-QLoJ4giHA8cXj7Pl6RI20A==/split_config.xxhdpi.apk

Note that app is split into 3 apk’s. Now decompile the main APK:

apktool -r d base.apk

This will extract APK contents to base. I then used jaxd-gui to decompile and inspect the (obfuscated) Java/smali source code.

Now we need to remove the code that checks the SSL certificates inside the X509TrustManagers. These can be found in base/smali_classes3/util/h/bs/a.smali and base/smali_classes3/util/ma/c.smali. Just return void early:

.method public checkServerTrusted([Ljava/security/cert/X509Certificate;Ljava/lang/String;)V
+ return-void

Next, inside the file base/smali/com/dnb/vipps/android/common/networking/h.smali there is extra certificate pinning code, we need to remove that:

    public final l.h a() {
h.a aVar = new h.a();
- aVar.a("api.vipps.no", "sha256/nTNUIFsFUeN3c+hYZzlJfKRIwoRtRDyY6sVGPEauJWY=");
- aVar.a("api.vipps.no", "sha256/O+qzlJSyoBSdYsSMceGYjjRqDxKVzUFjX0ymP+AC0f0=");
- aVar.a("*.api.vipps.no", "sha256/nTNUIFsFUeN3c+hYZzlJfKRIwoRtRDyY6sVGPEauJWY=");
- aVar.a("*.api.vipps.no", "sha256/O+qzlJSyoBSdYsSMceGYjjRqDxKVzUFjX0ymP+AC0f0=");
- aVar.a("ece46ec4-6f9c-489b-8fe5-146a89e11635.tech-02.net", "sha256/Fig44L8aGm+MmceSD17e15lV7pWlsDvgViYUSaj/mqM=");
- aVar.a("*.ece46ec4-6f9c-489b-8fe5-146a89e11635.tech-02.net", "sha256/Fig44L8aGm+MmceSD17e15lV7pWlsDvgViYUSaj/mqM=");
return aVar.b();
}

Next, the Android operating system provides an additional network security config. Add to application in base/AndroidManifest.xml:

<application ... android:networkSecurityConfig="@xml/network_security_config">

Then create a new file base/res/xml/network_security_config.xml with the contents:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<base-config cleartextTrafficPermitted="true">
<trust-anchors>
<certificates src="user" />
</trust-anchors>
</base-config>
</network-security-config>

Now recompile the modified APK:

apktool b base -o vipps-modified.apk

Now create a debug signing certificate for signing the APKs with:

echo y | keytool -genkey -v -keystore debug.keystore -storepass android -alias androiddebugkey -keypass android -keyalg RSA -keysize 2048 -validity 10000 -dname "cn=Mark Jones, ou=JavaSoft, o=Sun, c=US"

Re-sign all APKs using the same debug keystore just created:

echo android | apksigner sign --ks debug.keystore --out vipps-signed.apk vipps-modified.apk
echo android | apksigner sign --ks debug.keystore --out split_config.arm64_v8a-signed.apk split_config.arm64_v8a.apk
echo android | apksigner sign --ks debug.keystore --out split_config.xxhdpi-signed.apk split_config.xxhdpi.apk

Install all the new APKs to the Android phone

adb install-multiple vipps-signed.apk split_config.arm64_v8a-signed.apk split_config.xxhdpi-signed.apk

Run Charles proxy with SSL proxying and add the following domains to includes:

api.vipps.no
login.vipps.io
*.bankidnorge.no
*.bankidapis.no
*.bankid.no

From Charles, export the generated root certificate and download it to your Android phone and trust it for all apps/browser.

On the Android phone, launch

Start the app and start sniffing SSL traffic!

Next steps / observations

All Vipps related requests from startup, OTP, BankID auth and setting PIN as well as logging in:

https://api.vipps.no/app/security/v1/register
https://api.vipps.no/vipps-1.8/u-security/v1/generateotp/
https://api.vipps.no/vipps-1.8/u-security/v1/validateotp/
https://api.vipps.no/app/security/v1/token
https://api.vipps.no/bankid-auth/v1/bankid/url?login_hint=BID
https://login.vipps.io/vinx?login_hint=BID&requestId=...&session=...&target=https%3A%2F%2Fvipps.no%2F
https://login.vipps.io/bid/oidc/callback?state=vinx%3A...&session_state=...&code=...
https://login.vipps.io/vinx/callback?state=vinx%3A...&session_state=...&code=...
https://oidc.bankidapis.no/auth/realms/prod/precheck/auth?client_id=Vipps-BankID-Prod&login_hint=BID&nonce=...&redirect_uri=https%3A%2F%2Flogin.vipps.io%2Fbid%2Foidc%2Fcallback&response_type=code&scope=openid+profile+nnin_altsub&state=vinx%3A...&ui_locales=en
https://api.vipps.no/app/security/v1/token
https://api.vipps.no/app/security/v1/passphrase/srp/pin
https://api.vipps.no/app/security/v1/login/srp/init
https://api.vipps.no/app/security/v1/login/srp/pin/verify
https://api.vipps.no/vipps-1.8/u-security/v1/vinxlogin
https://api.vipps.no/vipps-1.8/a-profile/v1/details
https://api.vipps.no/vipps-1.8/customer/fetchblockedusers
https://api.vipps.no/vipps-1.8/merchantTransaction/v1/dueList
https://api.vipps.no/vipps-1.8/utility/v1/configParams
https://api.vipps.no/vipps-1.8/oneclick/v1/pendingRequests
https://api.vipps.no/vipps-1.8/invoice/v2/invoices_v3
https://api.vipps.no/vipps-1.8/a-profile/v1/details
https://api.vipps.no/vipps-1.8/payment/api/v1/agreement?includeCharges=true&status=PENDING
https://api.vipps.no/vipps-1.8/a-profile/v1/details
https://api.vipps.no/vipps-1.8/a-profile/v1/consents/consentcard
https://api.vipps.no/vipps-1.8/vippslogin/v1/vippslogintoken

Other reverse engineering tips

  • Dump resources (strings/translations): aapt dump resources base.apk | less

References

EDITLY - Slick, declarative command line video editing

Editly is a tool and framework for declarative NLE (non-linear video editing) using Node.js and ffmpeg. It allows you to easily and programmatically create a video from set of clips, images and titles, with smooth transitions between.

There is a simple command line for quickly assembling a video from a set of clips or images, or you can use it from Javascript!

demo

This GIF/youtube was created with this command: “editly commonFeatures.json5

See more examples here

Quickly identify minified React Native / web production stacktraces with stacktracify

Error in production? 😨

Stack trace looks like this? 😱

TypeError h is not a function. (In 'h()', 'h' is undefined) 
main.jsbundle:954:5353
main.jsbundle:112:423 p
main.jsbundle:112:1740
main.jsbundle:112:423 p
main.jsbundle:112:898 n
main.jsbundle:112:1273
main.jsbundle:50:205 c
main.jsbundle:50:1623 b
main.jsbundle:50:488 _
[native code] value
[native code] value

…perhaps from production from a minified web JS bundle or a React Native error report in a tool like Bugsnag, Sentry or Crashlytics

No source map yet uploaded to bugsnag? 😰

stacktracify takes a source map and a stack trace from your clipboard (or from a file) and outputs a readable stacktrace with proper line numbers for each line**

Example output:

TypeError h is not a function. (In 'h()', 'h' is undefined) 
at getAuthToken (logic/api.js:67:20)
at authRequest (logic/api.js:127:8)
at data (logic/SaveQueue.js:30:20)
at op (logic/SaveQueue.js:43:29)
at __callImmediates (node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:143:11)

How to do this? 😰

Copy the minified stack trace to clipboard, then run:

npm i -g stacktracify

react-native bundle --platform ios --entry-file index.js --dev false --bundle-output ios-release.bundle --sourcemap-output ios-release.bundle.map

stacktracify ios-release.bundle.map

Now sit back and relax, analyze the stack trace 😅 and fix the bug 😎💯

Read more: https://github.com/mifi/stacktracify

LosslessCut now on the Mac App Store and Microsoft Store

LosslessCut has become more and more popular recently, and I’ve gotten literally over a hundred of emails from people thanking me and saying LosslessCut is what they have been looking for for a long time. In these corona🍺times I’ve had lots of time to work on it and I have added a lot of new features and improved the user interface.

Many people have requested LosslessCut to be distributed through more channels to reach more people and ease installation and auto updating. Major operating systems also show a scary warning now, when trying to open an executable downloaded from the internet.

I decided to charge a small price for the apps distributed through the Microsoft and Apple stores, to cover the costs I’m paying for the Apple and Microsoft fees as well as the many hours I put into this very tiring work of dealing with release management. LosslessCut binaries downloaded directly from GitHub will of course always be free and open source. So if you want to support the work I do, you can either donate or purchase the app from the stores. Using the store version has the benefit of auto updates and less hassle with unverified executables that Mac OS and Windows always complains about these days when downloading files from the internet.

I hope you like it and I will continue improving LosslessCut 🙌

Get the new LosslessCut

The source code is freely available for anyone to look at, and if you are interested in seeing how the project is automatically built and deployed, check out this blog post.

Automated Electron build with release to Mac App Store, Microsoft Store, Snapcraft

I just released LosslessCut to stores on three different platforms Windows, Mac and Linux: Mac App Store, Microsoft Store, Snapcraft. I want to share the process and the hurdles I went through in this article.

I think the key to any well-maintained project is to have an automated release pipeline, or else it will be a hassle every time one wants to do bug fixes and improvements. I have now taken some time to set up automated build and signing for Mac OS (notarization) as well as release to the Mac App Store, Microsoft Store and Snapcraft. I have used many CI systems but I find that the new Github Actions are very fast and feature rich, and with its reusable action ecosystem it is quite easy to use. And all the setup and configuration is of course available open source for anyone to see here. See the actions in action(heh) here, as a free reference for anyone else needing to set up an automated build and release process of their Electron app.

Building and releasing for big stores like Microsoft and Mac App Store can be a true hassle and a nightmare that can take weeks to set up, however thanks to awesome projects like electron-builder and action-electron-builder, it is not really that bad and only took me a few days.

Here are the most valuable resources I used to set up everything:

Biggest issues

The things that I had the most trouble with were:

Getting all the correct metadata in place

This is mostly handled by electron-builder and can be seen in package.json, but some things did not give an error until a human looked at it during the review process.

Hardened runtime

In package.json, hardenedRuntime needs to be set to true for the mac platform, because notarized apps need to be hardened.

However for Mac App Store signed apps mas, it needs to be set to false.

Microsoft Store

Microsoft Store review

Reviewers complained about:

Mac App Store

The entitlements files

These files need to contain the correct entitlements in order to be able to open files and directories on the filesystem, as well as inherit these rights to the ffmpeg process. Without any entitlements, the app is not allowed to read/write files.

One thing I struggled a bit with is that Mac Store Apps need to have opened a file using the system open dialog before the app can read or write. In order to write to a directory, like LosslessCut does, I need to first present the user with an Open dialog to select which directory to output to. This seems to be a requirement in order for apps not to just write to any folder on the system.

App Icon

Icon needs to be .icns format and have 512 and 1024 sizes in it

Touch Bar API

Reviewers complained about what I believe is Electron using the Touch Bar API, but my app does not use it:

“If your app does not integrate Touch Bar functionality, please indicate this information in the Review Notes section for each version of your app in iTunes Connect when submitting for review”

So need to make sure to write this in Review notes for every release.

ffmpeg private API usage

Getting ffmpeg through App Store review is a bit tricky. The ffmpeg static build that I originally used has a lot of stuff built into them. One thing that App Store review complained about is a call to _SecIdentityCreate which I traced back to this file. But it could be disabled by the --disable-securetransport option. So I needed to build a custom ffmpeg without this flag.

nm ffmpeg | grep SecIdentityCreate
nm ffprobe | grep SecIdentityCreate

See also:

Other than that, with Electron 8, there was no other private API usages complained about.

When building ffmpeg on Mac OS, even with all options indicating it should include everything as static, it will still link .dylib files dynamically.

otool -L ffmpeg | grep /usr/local

I solved this by only building my own ffmpeg using GitHub Actions and stripping away all external dependencies like codecs (because we are only doing muxing operations which only requires ffmpeg core functionality.)

See also:

Building ffmpeg to support older Mac OS versions

On the first try I got an error report on older Mac OS X (10.13):

Command was killed with SIGABRT (Aborted): ffprobe -of json -show_format -i vid.mp4
dyld: lazy symbol binding failed: Symbol not found: ____chkstk_darwin
Referenced from: ffprobe (which was built for Mac OS X 10.15)
Expected in: /usr/lib/libSystem.B.dylib

This can be verified by running nm ffprobe | grep ____chkstk_darwin

In order to fix this I had to add -mmacosx-version-min=10.10 to --extra-clfags and --extra-ldflags

Then rebuild and verify that the ____chkstk_darwin symbol reference is gone.

Certificate creation/renewal

Go to certificates
https://developer.apple.com/account/resources/certificates/list
Create the following certs:

  • Developer ID Installer
  • Developer ID Application
  • Mac Installer Distribution
  • Mac App Distribution
  • Mac Development

Then dowload them and drag drop into Keychain access

Important: Now need to regenerate provisioning profile(s) after creating new certificates, and add the new profiles into the project (overwrite existing)

Then from Keychain access, select and export the following certificates to p12, and set MAC_CERTS and MAC_CERTS_PASSWORD in github secrets for the project:

  • Developer ID Installer
  • Developer ID Application
  • Mac Installer Distribution
  • Mac App Distribution

See https://github.com/samuelmeuli/action-electron-builder#code-signing

Mas build can no longer be run locally on a dev Mac. For running mas app locally, we need to create a separate provisioning profile for development, with the Developer Mac’s UUID and use mas-dev with that profile. See this issue.

See also:

Hiking to Coromandel Peak in Wanaka

Inspired by this post i wanted to hike Coromandel Peak in Wanaka, New Zealand. It is actually a shorter hike than going to Roys Peak, but a bit more adventurous. And you will have it all to yourself.

For anyone else who are thinking about doing the hike. I think it’s a lot easier to follow the Roys Peak track to approx 1100m altitude and then take right up to the ridge that goes down to Coromandel Peak. There is no trail up to the ridge, except for a few weak sheep tracks. Eventually when you get up on the ridge there will be a track there leading down to Coromandel Peak.

React Native: Smooth scroll to a specific element in a list (using hooks)

If you need to scroll to a specific View in a ScrollView, specified by a prop scrollToId to that page, do something like this:

const Component = ({ entities, scrollToId }) => {
const viewsRef = useRef({});
const scrollerRef = useRef();

useEffect(() => {
if (scrollToId == null) return;
const scrollToView = viewsRef.current[scrollToId];
if (!scrollToView) return;
scrollToView.measure((x, y, w, h) => scrollerRef.current.scrollTo({ x, y }));
}, [scrollToId]);

return (
<ScrollView ref={scrollerRef}>
{entities.map(({ entityId }) => (
<View key={entityId} ref={(ref) => { viewsRef.current[entityId] = ref; }}>
{/* ... */}
</View>
))}
</ScrollView>
);
}

Old non-hooks solution

render() {
const { entities, scrollToId } = this.props;

return (
<ScrollView ref={ref => {this.scrollerRef = ref; }}>
{entities.map((entity) => {
const scrollThis = scrollToId != null && scrollToId === entity.id;

return (
<View
key={entity.id}
ref={scrollThis ? (ref) => { this.scrollToRef = ref; } : undefined}
onLayout={scrollThis ? this.onLayout : undefined}
>
<Text>Some stuff to show from entity</Text>
</View>
);
})}
</ScrollView>
);
}

onLayout = () => {
this.scrollToRef.measure((x, y, width, height, pageX, pageY) => {
if (this.scrollerRef) this.scrollerRef.scrollTo({ x: pageX, y: pageY });
});
}

entities is assumed to be a list of something you want to render, with a unique prop id.

Install Java JDK on Mac OSX from dmg/pkg without admin/root

Oracle no longer provides compressed versions of JDK 😡 and require root to install (probably they also install an auto updater nagging you all the time)

Download .dmg from Oracle and put in a directory

cd directory-with-dmg

7z x jdk-8u181-macosx-x64.dmg

(alternatively, if you don’t have 7zip mount dmg and copy out .pkg)

pkgutil --expand "JDK 8 Update 181/JDK 8 Update 181.pkg" jdk-pkg-unpacked

tar xvf jdk-pkg-unpacked/jdk180181.pkg/Payload

Contents/Home is now the JDK root

mv Contents/Home ~/jdk180181

Now you can put JAVA_HOME="$HOME/jdk180181" and PATH="$PATH:$JAVA_HOME/bin" and start using it