In my opinion, Maya will only allow you to create the creature or character in question but doesn't really have support for real adaptive AI or anything like that.

The way such things are generally done is some programing genius creates the AI application and the artist creates the character and all the subsequant animations that go into it that the program will trigger.

I do not know of any 3D art application that provides what you seem to be looking for.