Kim's GTK/GNOME Submenu Project

This web site documents Kim's incipient effort (as of early April 2000) to improve submenu navigation within the GNOME desktop environment. This is Kim's very first Linux/GTK/GNOME programming effort, and he's likely to make some blunders, for which he invites both the indulgence and the gentle correction of the greater Linux community.

Since it is more than a little audacious to try to modify GTK without ever having written anything on top of it, this project will involve designing and coding a test app that will explore and demonstrate the new menu navigation functionality. (Much more on this later).

Please send feedback on this project to kimmccall@mail.com

The Problem

Items in hierarchical menus can pop up submenus. When the user sees a submenu, if it contains an item she wants to choose, she generally wants to move her cursor toward the desired submenu item. The submenu ought not to disappear when she does this. On the other hand, if she doesn't want to choose any item from this submenu, it ought to disappear when she moves to another parent-menu item. But we cannot read her mind to know whether or not she's interested in this submenu. The current GTK implementation of submenu navigation removes a submenu whenever the user strays out of the box of the current parent-menu item (unless she manages to get to the submenu first). This requires significant care, nontrivial mouse dexterity, and an annoyingly indirect path to most submenu items. We would like to do better.

Elements of a Solution

The "keep-up region"

Currently a submenu remains active only so long as the mouse remains within the box of the related parent-menu item. We can think of this box as a (primitive) "keep-up region." Intuitively, we want to expand this keep-up region to something more tolerant and generous. Here is an illustration of a possible keep-up region for a given menu item.

Figure 1. A keep-up region.

"Sticky" and "non-sticky" navigation modes

But simply using such a new keep-up region overcorrects the problem. We don't yet know that the user is interested in this submenu. She may simply be sliding down the parent menu on her way to the item that interests her. In Figure 1, imagine that she is moving down, intending to select something from the submenu of "Menu Item 4". She will find this maddeningly difficult. So we shouldn't honor this new keep-up region unless we have some reason to believe the user is interested in selecting an item from this submenu. Absent some such reason, submenus should appear and disappear just as they do now, using the parent-menu item's box as the keep-up region. This leads to the idea of switching between two different navigation modes, one in which the submenu is "sticky" and the other in which it is not. (This may not be the best nomenclature, as it may suggest that the mouse cursor itself is sticky or something like that. Suggested alternatives gladly entertained.) In sticky navigation mode we honor the expanded keep-up region, while in non-sticky mode we honor only the parent-menu item box.

Transitions between modes

Now the main problem becomes: "How do we know when the user is interested in a submenu?" Or in terms of implementation: "How do we know when to switch between navigation modes?" I'm happy to entertain suggestions. In moving from non-sticky to sticky mode, two heuristics occur to me. First, it seems to me a reasonable guess that a user will seldom move horizontally very far in the direction of a submenu unless she is interested in choosing something from it. So a reasonably tuned filter on mouse motion might show us when to enter sticky mode. Also, a user might reasonably expect a click in the parent menu item to mean "I'm really interested in you; don't go away." We might try implementing one or both of these. (Any other suggestions?) Transitioning back to non-sticky mode may be slightly more complex. Leaving the submenu for some other item on the parent menu surely ought to get us back into non-sticky mode. Also, if the user is still within the parent menu and moves horizontally significantly in the direction away from the child menu, we should give up the idea that she's interested in the child menu, at least until some further indication. [Since there is no clear visual feedback reflecting the current navigation mode, I think using a second mouse click to mean "now I'm no longer interested in this submenu" would be a terrible idea.] My guess is that implementing all these suggestions would guess correctly about the user's intentions at least 99% of the time, perhaps much more. My test app will allow users to experiment with each of the possible ways of getting from non-sticky to sticky mode. If other ways of getting back again are suggested, I'll incorporate them for experimentation, too.

Geometry of the "keep-up region"

Figure 1 gave a picture of one possible keep-up region. Others are possible and have been suggested. Essentially, the region is a hexagon with two opposite vertical segments, two semi-adjacent horizontal segments, and two semi-adjacent diagonal segments.

Figure 2. The full keep-up hexagon.

In principle, the length of the left-hand vertical segment could be reduced to zero, leaving a rotated "home-plate" pentagon.

Figure 3. The "home plate" keep-up pentagon.

It could also be simplified by reducing the lengths of the horizontal segments to zero, leaving a leftward pointing trapezoid.

Figure 4. The no-horizontals keep-up trapezoid.

Making both simplifications would leave a leftward pointing triangle.

Figure 5. The minimalist keep-up triangle.

Finally, there are plenty of possible intermediate values for the length of the horizontal segments.

Figure 6. A keep-up hexagon with intermediate-sized horizontal segments.

Among the extremely helpful suggestions I received after my initial posting of this project to the web several people suggested that the left edge of the keep-up region really does't belong at the left edge of the parent menu item, but rather much nearer the mouse cursor location. So, if we use the full hexagon shape, this suggestion makes our region look more like the following.

Figure 7. A keep-up hexagon with left edge determined by mouse cursor location.

And if we modify this to have a left edge of zero length we get a smaller "home plate" region.

Figure 8. A keep-up "home plate" with left edge determined by mouse cursor location.

My own intuition is that these last two geometries are the only ones that really make much sense (where would I be without your suggestions), but my test app will allow users to experiment with all of the above proposals to see what feels best.

Hysteresis

Finally, I have been alerted by my web collaborators to the importance of hysteresis, or "lag of effect," in producing a pleasing interface. [For a nice (and highly pro-MacOS) discussion, see http://www.mackido.com/Interface/hysteresis.html] A user's interest can often be determined by her speed of action or by her slowing down or pausing. Also, when actions on the screen happen too quickly there can be a distracting flashing effect. One obvious place to introduce hysteresis is in the rapidity with which submenus appear when you roll over their parent menu items. A slight delay will allow you to move to the next item without flashing up an unwanted submenu. [GTK may already be doing something like this, although I'm not certain. I see some "timer"s in the code, but I haven't yet determined what they're used for.] More central to our current project is the question of whether we shouldn't use some kind of hysteretic techniques to govern the transition between sticky and non-sticky modes and perhaps even the shape of the keep-up region.

Here's a proposal to consider. We wait until a user either (a) slows/pauses/spends-time in a menu item or (b) clicks in it, before we even bring up the submenu. Now, since we believe the user is probably interested in this submenu, we enter sticky mode just as soon as we bring up the submenu. There's no submenu-is-showing-but-not-sticky state. We supplement this with a dynamic keep-up region of the general geometry shown in Figure 7 or Figure 8. The left-hand edge of our keep-up region is adjusted hysteretically according to the recent movements of the cursor. Properly tuned, this would allow for slight or slow leftward movements without erasing the submenu.

There may be a problem with my approach that occurred to me when reading the above-mentioned (hysteresis.html) link. Consider Figure 8. Suppose the user happens to move very slowly in the direction of "Submenu item 6". What does she want or expect? My guess is that if she moves slowly enough she doesn't actually have in mind selecting that menu item, but rather is just sloppily moving on to "Menu Item 4." But that's a guess. Maybe that's what 80% of users mean, and 20% are, in fact, intentionally moving toward "Submenu item 6" but just doing so slowly. Should we erase the submenu to best serve the 80%, or should we keep it up to please the 20%? Given that losing the submenu is much more annoying than leaving it up until the user moves straight down or to the left some, we might stick with our current approach. The alternative is to require that the user move somewhat quickly toward the submenu in order to keep it up. And this may be what would be expected by a user who has intuitively noticed our hysteretic UI, and may be implemented by hysteretic techniques. My intended test app is getting pretty heavily featured, but it would be good to be able to experiment with these questions.

My Preferred Solution

Thinking it over for a whole couple minutes, it seems to me that:

  1. the submenu should be raised when the user pauses, clicks, or spends much time (e.g. moving horizontally) in the parent menu item (pausing is, of course, a subcase of spending much time);
  2. the submenu should stay alive as long as either
    1. the user stays within the parent menu item,
    2. the user stays within the submenu, or
    3. the user stays within the keep-up region and is moving with a decent velocity whose horizontal component is in the direction of the submenu;
  3. the keep-up region should be shaped as shown in Figure 7 or Figure 8, and it should be adjusted dynamically (and hysteretically) to track the motion of the cursor.

Simplifications:

If our hysteretic delay is small enough, we probably don't actually need to do anything special for clicks in the parent menu item. Assuming that the user hangs around a bit after clicking, the submenu will come up without any special provisions.

Given the requirement in 2.2 for positive horizontal velocity, there may be no practical difference between a dynamic and a static keep-up region, and a static Figure 2 may serve as well as a dynamic Figure 8 with less computational load. This would also simplify our code. I'll be analyzing this further. Requirement 2 above seems to imply a double-keep-up-region solution. See Figure 9. The menu stays up as long as the user is within the blue outline or both within the pink outline and moving with reasonable speed in a reasonable direction (i.e. toward the submenu).

Figure 9. A double keep-up region.

While this drawing may look more complicated than the others, I think the user experience and implementation will be simpler. A little deeper analysis may even reveal that the pink region may be replacable by a constraint like "keep the submenu up just as long as the user is moving within the parent menu with a decent velocity whose horizontal component is in the direction of the submenu."

Unresolved Questions that have occurred to me

A serious possible problem is that menubars and menus seem to be implemented with the same code, so that menubars are practically just big horizontal menus. I, for one, don't really expect the stickiness behavior of a menubar to be like that of a menu. I don't think of them as the same thing at all. I'm afraid that modifying the behavor of menus as we want to here will generate fairly bizarre feeling behavior in menubars. [Need to think this through more, but wanted to note it.]

How much horizontal motion is enough to trigger a change between navigation modes? I should incorporate a GtkScale into the test app to tune this.

Make sure that any clicks in the parent menu on some other item (within the keep-up region, but outside the parent-menu item area) result in the correct behavior. The submenu should disappear, and if the newly selected item is not a parent of a submenu, it should be invoked. Otherwise its submenu should appear, and if we use clicking as a way of stickifying the submenu, we should go into sticky mode.

Requests for Feedback

Do you notice any glaring errors I'm making here? Am I completely out to lunch about how to go about developing for Linux/GTK? Do you see any design errors? Any omissions of important considerations? Again, please send any feedback to kimmccall@mail.com.

Plan of Attack

Develop test app

[TODO: Fill this out some more]

I am developing a reasonably featureful application to validate my code and to experiment with the various possible keep-up region geometries and especially the different ways of transitioning between sticky and non-sticky navigation modes or tuning hysteresis. Here's a glimpse of how it looks right now. The picture of the menus is just an illustration of the keep-up region as determined by the widgets to its left. At the bottom you can choose which of the methods of moving from sticky to non-sticky mode will be enabled. (I must add a "pure hysteresis" option to this group and a "scale" widget or two to adjust hysteresis delays or keep-up velocity values.) And I have yet to write any of the GTK code to actually get the menus to act the way the widgets say they should.

Figure 7. UI of my test app (so far)

Experiment myself

Before I release my test app to the greater GNOME community, I should see what I can learn from it myself.

Release test app to the web

In case there are people out there who would like to experiment with my modified menus and play with the various parameters about which decisions must be made (one or two ways of getting into sticky mode; geometry of keep-up region), I'll make my test app available for all to play with. This will also give me a good chance for early code review for quality and to bring me into line with gtk standards.

Get feedback

Hopefully, people will respond by trying it out and letting me know how it can be improved.

Incorporate feedback

I'll try to act on all reasonable suggestions. I'll also try adding to this web site at least a summary of our discussion of interesting issues related to this project.

Repeat previous three steps as necessary

Submit patch

I guess all that's left to do is to go through the standard submission process.