My basic approach was to use a model close to how actual tentacles work. If the basis for their movement was unrealistic, no amount of tweaking would get them to feel quite right. Compared to the limbs of creatures with bones, tentacles have a lot more degrees of freedom, and can also compress and stretch. The model I chose is as follows:
A tentacle consists of about 20 segments. Each segment is attached to the previous one with a joint. There are two muscles in each segment, which can independently shorten, up to about half of their original relaxed length. The angle of a segment is determined by the angle of the previous segment plus the angle created by the differing lengths of the muscles.
This model has two advantages: it's close to how actual tentacles work, and the behaviour of the tentacle comes down to the behaviour of each muscle, which can be represented as a single compression value.
I programmed this model and started out with a simple rule that made the muscles relax by default, straightening the tentacle and extending it to its maximum length. Then, to test it, I turned the cursor into a kind of an "electroshock" tool which would contract nearby muscles with a mouse click. This let me check that I'd wired up the model correctly, and that the basic movement of the tentacle was fluid.
You can try this out here: click on the area below to manually contract the tentacle's muscles. (With all of these examples, feel free to look at the source code for implementation details.)
My basic idea for muscle behaviour was as follows: each segment should attempt to position itself to be at a certain angle relative to the target point. Segments at the start of the tentacle should point straight towards the target, while segments towards the end should aim to be at an increasing angle towards it, still coming closer, but encircling it. Each muscle would figure out in which direction its segment needed to rotate and expand or contract accordingly.
And this already worked pretty well! The tentacle would head towards the target and then curl around it. The main problem was that it would not actually touch it. For the next iteration, I modified the behaviour so that the tip of the tentacle would again want to point straight towards the target.
I also adjusted the compressibility of the segments, making the segments at the base stiffer and the ones near the tip more flexible. Once that was in place, the tentacle was able to get within a fairly short distance of most target points.
In-game, the tentacles of the sky squid will seize individual crew members and deliver them to the maw of the creature. I programmed this in next, giving the tentacle targets that alternated between random points and a fixed point next to the tentacle's base where the mouth would be.
There were still some cases where the tentacle would get stuck, each segment having locally optimized its position but the tentacle as a whole failing to connect, vibrating as it tried to optimize further. Effectively, the tentacle was stuck in a local maximum. No small change would improve its position. So I put in an extra rule that if the tentacle took more than about two seconds to find its target, it would temporarily relax the muscles in its base, causing a large movement that would get it unstuck.
The tentacle's behaviour is now reliable and looks cool, and is ready to be integrated into the game.
