I made an example GitHub Action workflow that will automatically push all files in a specified directory to a different GitHub repo using a GitHub deploy key. By enabling GitHub Pages for the destination repo, you can serve content dynamically generated by a GitHub action workflow/job.
I previously attempted to do the same with an early version of the Vipps (Norwegian easy payment app), but the app has changed a lot since then. I decided to try again with the current version, and I learned that it has implemented quite a few new countermeasures preventing the old method from working.
The process
First download Vipps app on an Android phone, then connect it and copy the app over USB:
Note that app is split into 3 apk’s. Now decompile the main APK:
apktool -r d base.apk
This will extract APK contents to base. I then used jaxd-gui to decompile and inspect the (obfuscated) Java/smali source code.
Now we need to remove the code that checks the SSL certificates inside the X509TrustManagers. These can be found in base/smali_classes3/util/h/bs/a.smali and base/smali_classes3/util/ma/c.smali. Just return void early:
.method public checkServerTrusted([Ljava/security/cert/X509Certificate;Ljava/lang/String;)V + return-void
Next, inside the file base/smali/com/dnb/vipps/android/common/networking/h.smali there is extra certificate pinning code, we need to remove that:
I’m stuck at the salting/hashing algorith of the PIN when creating/authenticating with a PIN. Seems like the app uses the Secure Remote Password protocol (SRP)
Seems like it uses JSON Web Key for the initial /app/security/v1/registerdeviceKeys call.
Editly is a tool and framework for declarative NLE (non-linear video editing) using Node.js and ffmpeg. It allows you to easily and programmatically create a video from set of clips, images and titles, with smooth transitions between.
There is a simple command line for quickly assembling a video from a set of clips or images, or you can use it from Javascript!
TypeError h is not a function. (In 'h()', 'h' is undefined) main.jsbundle:954:5353 main.jsbundle:112:423 p main.jsbundle:112:1740 main.jsbundle:112:423 p main.jsbundle:112:898 n main.jsbundle:112:1273 main.jsbundle:50:205 c main.jsbundle:50:1623 b main.jsbundle:50:488 _ [native code] value [native code] value
…perhaps from production from a minified web JS bundle or a React Native error report in a tool like Bugsnag, Sentry or Crashlytics
No source map yet uploaded to bugsnag? 😰
stacktracify takes a source map and a stack trace from your clipboard (or from a file) and outputs a readable stacktrace with proper line numbers for each line**
Example output:
TypeError h is not a function. (In 'h()', 'h' is undefined) at getAuthToken (logic/api.js:67:20) at authRequest (logic/api.js:127:8) at data (logic/SaveQueue.js:30:20) at op (logic/SaveQueue.js:43:29) at __callImmediates (node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:143:11)
How to do this? 😰
Copy the minified stack trace to clipboard, then run:
LosslessCut has become more and more popular recently, and I’ve gotten literally over a hundred of emails from people thanking me and saying LosslessCut is what they have been looking for for a long time. In these corona🍺times I’ve had lots of time to work on it and I have added a lot of new features and improved the user interface.
Many people have requested LosslessCut to be distributed through more channels to reach more people and ease installation and auto updating. Major operating systems also show a scary warning now, when trying to open an executable downloaded from the internet.
I decided to charge a small price for the apps distributed through the Microsoft and Apple stores, to cover the costs I’m paying for the Apple and Microsoft fees as well as the many hours I put into this very tiring work of dealing with release management. LosslessCut binaries downloaded directly from GitHub will of course always be free and open source. So if you want to support the work I do, you can either donate or purchase the app from the stores. Using the store version has the benefit of auto updates and less hassle with unverified executables that Mac OS and Windows always complains about these days when downloading files from the internet.
I hope you like it and I will continue improving LosslessCut 🙌
The source code is freely available for anyone to look at, and if you are interested in seeing how the project is automatically built and deployed, check out this blog post.
I just released LosslessCut to stores on three different platforms Windows, Mac and Linux: Mac App Store, Microsoft Store, Snapcraft. I want to share the process and the hurdles I went through in this article.
I think the key to any well-maintained project is to have an automated release pipeline, or else it will be a hassle every time one wants to do bug fixes and improvements. I have now taken some time to set up automated build and signing for Mac OS (notarization) as well as release to the Mac App Store, Microsoft Store and Snapcraft. I have used many CI systems but I find that the new Github Actions are very fast and feature rich, and with its reusable action ecosystem it is quite easy to use. And all the setup and configuration is of course available open source for anyone to see here. See the actions in action(heh) here, as a free reference for anyone else needing to set up an automated build and release process of their Electron app.
Building and releasing for big stores like Microsoft and Mac App Store can be a true hassle and a nightmare that can take weeks to set up, however thanks to awesome projects like electron-builder and action-electron-builder, it is not really that bad and only took me a few days.
Here are the most valuable resources I used to set up everything:
This is mostly handled by electron-builder and can be seen in package.json, but some things did not give an error until a human looked at it during the review process.
Hardened runtime
In package.json, hardenedRuntime needs to be set to true for the mac platform, because notarized apps need to be hardened.
However for Mac App Store signed apps mas, it needs to be set to false.
Microsoft Store
Microsoft Store review
Reviewers complained about:
Screenshot did not have Windows style border, but Mac style. Easy fix.
Notarization happens only for downloadable binaries after they are built (not for Mac App Store submitted binaries). Therefore the mac target (but not mas).
Notarization is uploading the .app to Apples servers so they can verify it. Uploads the mac binary (has to be signed with Developer ID Application identity) https://developer.apple.com/forums/thread/128772
Mac App Store
Signing for mas and mas-dev uses identities Apple Distribution and Apple Development respectively. (this is new for Xcode 11, used to be Mac App Distribution and Mac Development). Finally pkg (which is uploaded to Mac App Store) is signed with 3rd Party Mac Developer Application identity. https://github.com/electron-userland/electron-builder/issues/2513
The entitlements files
These files need to contain the correct entitlements in order to be able to open files and directories on the filesystem, as well as inherit these rights to the ffmpeg process. Without any entitlements, the app is not allowed to read/write files.
One thing I struggled a bit with is that Mac Store Apps need to have opened a file using the system open dialog before the app can read or write. In order to write to a directory, like LosslessCut does, I need to first present the user with an Open dialog to select which directory to output to. This seems to be a requirement in order for apps not to just write to any folder on the system.
App Icon
Icon needs to be .icns format and have 512 and 1024 sizes in it
Touch Bar API
Reviewers complained about what I believe is Electron using the Touch Bar API, but my app does not use it:
“If your app does not integrate Touch Bar functionality, please indicate this information in the Review Notes section for each version of your app in iTunes Connect when submitting for review”
So need to make sure to write this in Review notes for every release.
ffmpeg private API usage
Getting ffmpeg through App Store review is a bit tricky. The ffmpeg static build that I originally used has a lot of stuff built into them. One thing that App Store review complained about is a call to _SecIdentityCreate which I traced back to this file. But it could be disabled by the --disable-securetransport option. So I needed to build a custom ffmpeg without this flag.
Other than that, with Electron 8, there was no other private API usages complained about.
Building a signable ffmpeg without dynamic links
When building ffmpeg on Mac OS, even with all options indicating it should include everything as static, it will still link .dylib files dynamically.
otool -L ffmpeg | grep /usr/local
I solved this by only building my own ffmpeg using GitHub Actions and stripping away all external dependencies like codecs (because we are only doing muxing operations which only requires ffmpeg core functionality.)
On the first try I got an error report on older Mac OS X (10.13):
Command was killed with SIGABRT (Aborted): ffprobe -of json -show_format -i vid.mp4 dyld: lazy symbol binding failed: Symbol not found: ____chkstk_darwin Referenced from: ffprobe (which was built for Mac OS X 10.15) Expected in: /usr/lib/libSystem.B.dylib
This can be verified by running nm ffprobe | grep ____chkstk_darwin
In order to fix this I had to add -mmacosx-version-min=10.10 to --extra-clfags and --extra-ldflags
Then rebuild and verify that the ____chkstk_darwin symbol reference is gone.
Certificate creation/renewal
This also needs to be re-done when certificates expire.
Then dowload them and double click to load into Keychain access.
Important: Now need to regenerate the provisioning profile after creating new certificates (select the new Apple Distribution certificate), then add the new .provisionprofile file into the project (overwrite existing). Similarly for for development provisioning profile.
Then from Keychain access (use My Certificates tab to get certficates with keys), select and export the following certificates to p12, and set MAC_CERTS and MAC_CERTS_PASSWORD in github secrets for the project (randomly generate the password):
Apple Distribution (mas signing for App Store)
3rd Party Mac Developer Installer (for wrapped .pkg for App Store)
Developer ID Application (needed for notarization)
Mas build can no longer be run locally on a dev Mac. For running mas app locally, we need to create a separate provisioning profile for development, with the Developer Mac’s UUID and use mas-dev with that profile. See this issue.
Troubleshooting
Testing signing locally
Make sure you have these in your keychain: Apple Development, Apple Distribution.
Inspired by this post i wanted to hike Coromandel Peak in Wanaka, New Zealand. It is actually a shorter hike than going to Roys Peak, but a bit more adventurous. And you will have it all to yourself.
For anyone else who are thinking about doing the hike. I think it’s a lot easier to follow the Roys Peak track to approx 1100m altitude and then take right up to the ridge that goes down to Coromandel Peak. There is no trail up to the ridge, except for a few weak sheep tracks. Eventually when you get up on the ridge there will be a track there leading down to Coromandel Peak.
Oracle no longer provides compressed versions of JDK 😡 and require root to install (probably they also install an auto updater nagging you all the time)
Download .dmg from Oracle and put in a directory
cd directory-with-dmg
7z x jdk-8u181-macosx-x64.dmg
(alternatively, if you don’t have 7zip mount dmg and copy out .pkg)
