You have stolen the checking program for the CSAW Challenge-Response-Authentication-Protocol system. Unfortunately you forgot to grab the challenge-response keygen algorithm (libchallengeresponse.so). Can you still manage to bypass the secure system and read the flag?
The binary is an ELF 32-bit binary that accepted socket accepted input from stdin(and was running using xinetd or something similar). Looking at the binary we can see that there is a call to _fillChallengeResponse from the missing shared object file, that is probably setting two global variables(renamed as g1 and g2).
The program then enters the following loop, where it reads a byte from the user, ands it against 0xF0, compares it against 0xA0, 0xE0 and 0x80 -- and performs actions accordingly.
We can see that our input char is and'd with 0x0F, left shifted twice(multiply by 4), added to the base address of g2 and 4 bytes are read back. Essentially, given an index, 4 bytes can be dereferenced starting at an offset from g2.
This function is interesting as it computes the computes the product of the contents of a global integer array and prints out the flag if the product is 1. Needless to say, each element of the global array is initialised to 0.
The input character is and'd with 0xF, left shifted by 2 and used as an index to deref addresses with g1 as the base. It then reads 4 bytes from the user and checks its contents against the deref address; if they match they set the globalarray[pos] = 1 where pos is calculated as inputchar & 0xF.
Armed with this information, we can try and write a python script that supplies values to trigger case 160, leaks values, triggers case 224, uses the leaked value to set a position in the global array to 1. Finally, we can trigger case 128 to read the key.
One could create a libchallengeresponse.so file with a _fillChallengeResponse function that could set g1 and g2, so that we could try debugging the saturn binary against our exploit.