Can I install apps via TestFlight for Automation?

No, TestFlight is currently not supported on App Automate. Apple places a restriction on the number of devices that can be used to sign in using TestFlight. Once the user breaches this limit, app installation via TestFlight is blocked. Users will have to manually login into their Apple account and delete those devices to resolve this issue. This quickly becomes an unmanageable process with automated testing on a public cloud. 

Also, TestFlight requires two-factor authentication which can't be disabled on newer Apple accounts. This also poses another challenge in supporting TestFlight.