Here is a figure that describes the overall organization of Linux-PAM:
+----------------+
| application: X |
+----------------+ / +----------+ +================+
| authentication-[---->--\--] Linux- |--<--| PAM config file|
| + [----<--/--] PAM | |================|
|[conversation()][--+ \ | | | X auth .. a.so |
+----------------+ | / +-n--n-----+ | X auth .. b.so |
| | | __| | | _____/
| service user | A | | |____,-----'
| | | V A
+----------------+ +------|-----|---------+ -----+------+
+---u-----u----+ | | |
| auth.... |--[ a ]--[ b ]--[ c ]
+--------------+
| acct.... |--[ b ]--[ d ]
+--------------+
| password |--[ b ]--[ c ]
+--------------+
| session |--[ e ]--[ c ]
+--------------+
By way of explanation, the left of the figure represents the application; application X. Such an application interfaces with the Linux-PAM library and knows none of the specifics of its configured authentication method. The Linux-PAM library (in the center) consults the contents of the PAM configuration file and loads the modules that are appropriate for application-X. These modules fall into one of four management groups (lower-center) and are stacked in the order they appear in the configuration file. These modules, when called by Linux-PAM, perform the various authentication tasks for the application. Textual information, required from/or offered to the user, can be exchanged through the use of the application-supplied conversation function.
If a program is going to use PAM, then it has to have PAM functions explicitly coded into the program. If you have access to the source code you can add the appropriate PAM functions. If you do not have access to the source code, and the binary does not have the PAM functions included, then it is not possible to use PAM.