You can explicitly revoke the distribution certificate for an application in which case the build will become invalid.
When the next commit to your project repository is made, Windmill will distribute a new build.
Tip: Press ⎇⌘R to run Windmill with the existing source code.