Custom Dialog windows can be viewed as Internet Explorer instances, which allows the developer to handle and customize them accordingly.
In order to do this, we must first take a look at the dialog as a web page. If we run the Custom Dialog Process through the Console, we may use the Process Designer to attach to the open dialog (as an IE page with the title "Custom Dialog"), and use the "Get Details of Web Page" action to extract the source code. Then, we may save the source code in a new .html file and open it using any browser. Check this forum post for a script which extracts the HTML code of a Custom Dialog in the above way.
The last step will be to set this process to run when the Dialog appears. Simply add a "Start Process" Action before the "Display Custom Dialog" one, and set this secondary Process to run. The dialog will appear, the secondary Process will immediately run its code on it and terminate, and the dialog will now behave as you wish.
Please check this forum post for a sample Process which was built using this technique.