acm sigcomm

ACM SIGCOMM Student Mentoring Column
Aditya Akella, UW—Madison
 
BIN Problems
 
 
As some of you may know, I gave a talk at CoNEXT 2014 titled “On future-proofing networks” (see [1] for slides). While most of the talk was focused on my past research projects, I spent a bit of time talking about the kind of problems I like to work on. I got several interesting questions about the latter, both during the talk and in the weeks following the talk.
 
Given this, I thought I would devote this column (and perhaps parts of future columns) to putting my ideas on problem selection into words. I suspect what I write below will generate more questions than answers in your mind. Don’t hold your questions back, though! Write to me at [email protected]!
 
BIN problems
 
I like to work on BIN problems, which are research problems that are: Big in scope; Important in terms of their potential for intellectual or practical impact; and, Nebulous in terms of how to approach them. It is hard to precisely define BIN problems, but here are examples of BIN problems that I worked on that may give you a sense of what I’m talking about: How to design an evolvable Internet architecture? How to model and mitigate network complexity? And, how to exercise software-defined control over network functions?
 
Why am I attracted to BIN problems?
 
Pick one of the examples above, say, control over network functions (NFs). The scope is Big—different networks use different subsets of NFs, and there are many different kinds of NFs out there. The problem appears Important—the industry is starting to think about this. From a research standpoint, there appear to be few studies that offer systematic answers. Thus, the potential for impact on both practice and research exists.
 
The Big-ness of scope and the Nebulous-ness of approach mean, naturally, that the “field is wide open”, i.e., there is no immediately obvious answer here. There are many places to start approaching this problem, and many potential ideas/issues to consider. For example, do we simply control routing to NFs? Does it make sense to control internal state? How about dynamic context? NF logic? Algorithms? How to deal with NF diversity?
 
I hope this example gives you a sense as to why I think BIN problems are so cool: they keep your mind constantly jogging. And, if you succeed, the pay-off can be huge. In what follows, I will try to address two questions that may arise regarding such problems.
 
Are BIN problems risky?
 
Some of you may wonder if this is too much to take up. Where do you start? Which thread(s) do you explore first? What if your exploration fails? It may also be the case that the BIN problem you picked doesn’t seem “hot” (this isn’t exactly true for network functions—there’s a lot of buzz surrounding them these days—but this concern may arise more naturally for other topics such as network complexity, or the Internet’s evolvability).
 
I would argue that the situation is quite the contrary. First, note that once you’ve picked a reasonable starting point (control over internal NF state in the example above), you’ve rooted a version of the problem, and this immediately makes things less vague. From this point on, it is like any other research problem in terms of the likelihood of finding something tangible. But the Importance aspect means that whatever you find has a big likelihood of having impact. The impact could be intellectual in nature—your work could spur others to build similar, perhaps better, systems but your system will be the first. Or, you may hit the nail on its head right away, in which case you could directly impact practice.
 
As you explore, things may turn out to be not what you started out hoping for. But the thing about BIN problems is that there’s enough room for refinement: e.g., maybe you started out by casting too wide a net around NFs, and your initial experience taught you that focusing on, say, distributed control across multiple copies of an NF is still Big and Important, and now it’s also less Nebulous.
 
Finally, BIN problems naturally lead to bodies of work. The many threads to explore at each starting point, and the different potential starting points themselves, often map on to distinct but related research projects/problems. In-depth exploration of one particular starting point and associated threads often gives you enough context to explore in a more focused fashion other starting points and associated threads.
 
How to work on BIN problems?
 
Crucially, as a student, how do you know something is Important? Also, you may not be able to judge if the scope is Big enough, or perhaps it too Big. And related to this is the question of whether your problem statement is clear enough for you to start coming up with meaningful solutions.
 
You don’t have to fight these battles alone. Working on BIN problems is not something you could or would want to do sitting behind locked doors.
 
To clarify scope and to pin down a problem statement, I strongly encourage you to talk to your advisor about your idea (in case you haven’t noticed, advisors love to hear bold and fresh ideas from you more than having you simply “do what you are told”). Talk to others in your research group, including both professors and students. Then, talk to others in the broader networking area, perhaps when you bump into them at conferences; also, consider inviting yourself to go give a talk some place and chat 1-on-1 with people there about your BIN idea (your advisor could be of help in setting up such a rendezvous). In particular, talking to practitioners is quite valuable here, and this set includes your local campus network operators, researchers at industry research labs, engineers at major online service providers, etc.
 
The initial reaction you get may make you feel like you are wasting your time. I have in the past had people tell me I was “crazy” to work on something, or that what I was working on was “not relevant”. But this is actually constructive feedback in the sense that what may be happening is that your pitch for the problem needs refinement. Go back to your notepad, and come up with a better framing of your problem! And then repeat the process above.
 
Approaching BIN problems doesn’t necessarily have to be top-down, i.e., you don’t have to start with a BIN problem. Often, you may find yourself working on a concrete and interesting problem. Perhaps you have a couple of papers on that already. It is a good idea to stop there and ask, “Is the scope of what I am working on bigger than what I’ve looked at in my 1-2 papers so far?” And, you should be open to the fact that broadening the scope may require a different approach (one that is necessarily nebulous at the beginning) than what you’re currently exploring.
 
Working on BIN problems can take time. In particular, getting the right initial intuition nailed down could take a year, perhaps more. All the things I mentioned above—talking to people, gathering feedback, going back to the notepad—can appear to drag things out, too. But once you have the right intuition, and given the experience you’ve gathered along the way, you’re going to be unstoppable!
 
References
 
[1] “On future-proofing networks”, http://www.cs.wisc.edu/~akella/talks/conext-keynote-web.pptx

Hello Everyone,

Hamed Haddadi, the Information Service Director for SIGCOMM, has found some of my posts over at "How to Do Great Research" quite worthwhile to read, and has suggested that I cross-post some of the content on the SIGCOMM blog.  I'll point you over to the blog for the full content, but I've posted a teaser below.

The genesis of the blog is a course for incoming Ph.D. students that Professor Alex Gray and I designed at Georgia Tech and have been teaching for more than five years.  To our surprise, a lot of helpful information on the Ph.D. is "out there", but it's not really consolidated in one place.  We began collecting helpful information and filling in the gaps with our own material; this fall, as much of the material has coalesced, I am putting the material into written form on a blog.  You'll be able to read posts such as the one below throughout the fall term. (I expect the flood of posts this fall may become a trickle in the future, but hopefully once the initial information is seeded, we'll also enrich the information we have with guest posts.)  The course has a variety of twists and turns that we devised, such as assignments to generate cross-disciplinary ideas, time management exercises, a mock program committee, etc.  If anyone is interested in more of the course materials or wants to talk to me more about this concept, I'm all ears.

Thanks, and happy reading!

Nick Feamster

-------------------------

Managing Your Advisor (cross-posted from How to Do Great Research)

With the new academic term almost upon us, several of my students started to put together a list of practical advice for incoming students—including various niceties such as how to gain access to the lab, how to get accounts, how to submit reimbursements, and so forth.  I wanted to contribute to the list of advice, and I figured I could offer some value by giving advice to new students about how to gain traction on their research as quickly as possible.  This post is the first in a series of a few posts on that topic; in this post, I will cover the topic of managing your advisor.

The notion of an advisor is an interesting concept for many new Ph.D. students.  Incoming graduate students typically have one of two backgrounds: some come straight from undergraduate studies (and, hence, may have never had a manager or a boss overseeing their career); others have spent some time in the workforce and have decided to return to the university and begin a career in research (and, hence, have some notion of what it is like to have a manager).  An advisor-student relationship is unique, though, and will be a new experience for both types of incoming students.  The relationship is similar to a manager relationship, but has several differentiating features.  First, your advisor is often a collaborator on equal footing.  Although an incoming Ph.D. student is not (yet) a peer of his or her advisor, the goal is that by the end of the Ph.D. process, the student and advisor will be peers.  In this sense, the Ph.D. is a true apprenticeship.  My students don’t work for me; they work with me.  Second, your advisor is not a manager in the strict sense, but is literally an advisor: You are in control of shaping your own graduate career, from what you choose to work on to who you work with.  Your advisor should be a catalyst and facilitator for your success and should not be treating you as an employee or “hired labor”.  Although some research contracts have deliverables, you should be suspicious of any advisor who wants to constantly hold you to tight deliverables, as it will constrain your autonomy and creativity; that type of advisor will ultimately be more like a manager, and you can find plenty of managers in industry who will pay you a much higher salary.  If you find that your advisor is bossing you around or restricting your autonomy or creativity, change advisors as soon as possible.

In any advisor or managerial context, it is important to recognize the importance of “managing up”.  While there may be strategic reasons to do this in any context, the most important reason to learn how to manage your advisor is to make the most of your graduate career.  Many things compete for your advisor’s attention—papers, grants, proposals, teaching, committees, other students, outside opportunities, etc.  At the same time, everyone’s Ph.D. experience is unique, and it is incumbent on you to work with your advisor to help you define your own trajectory and also to create a working relationship that works for both of you.

In my seven years as an advisor, I have learned a few things about my working style.  Here is some of the advice I have offered my students about how to manage me.  Many of these tips may be useful in general for other Ph.D. students who want to help build a better relationship with their advisor and help get the most out of their graduate careers:

• Ask your advisor for what you need.  Want to attend a conference, get an introduction to a senior colleague in the field, buy a book or other equipment, find an internship, get a travel grant, or something else?  Be proactive.  The answer will be “yes” more often than you think .

• Scheduling meetings. I have a Google calendar that I share with all of my students.  If a meeting or event is not on my calendar, the student should assume that the meeting is not happening, even if the meeting has been discussed (and agreed on!) in the hallway.  There is no way to keep track of hallway discussions for scheduling and they are quickly forgotten.  Though it’s not strictly necessary, I advise my students to consider sending a reminder/minutes/confirmation before the meeting; this relates to the point below on making meetings count.  Scheduling meetings sometimes can generate an explosion of email—this is a recipe for disaster and ensuring that you never get to meet your advisor (see below on email); if scheduling is proceeding slowly, limit the email thread to 1-2 emails before suggesting a meeting invitation by Google calendar.  If all else fails, send a meeting invitation during an open slot; in the worst case, your advisor will react by moving it to a time that works (it is on the calendar and thus can no longer be deferred indefinitely).

• Try to meet your advisor once a week, even if you think you have nothing to talk about.  Make an effort to schedule a meeting once a week, even if the meeting is short; in my experience, I have found that sometimes even a ten-minute meeting with a student can make a huge difference for working around a mental block or changing an approach to a problem.  Do not assume that a meeting cannot happen simply because your advisor is not in town.  Short meetings by Google hangout are often very handy.  In fact, throughout the summer of 2013, I was rarely at Georgia Tech; many of my students actually found it easier to meet me when I was traveling because I wasn’t being constantly bombarded by things related to the daily drumbeat at the university (e.g., committee meetings, interruptions from admins, teaching, etc.).  Consider having a meeting even if you think there’s nothing to report.  You may find you are stuck in a rathole, and you may not even realize it.  You should be particularly worried if you have spent 2-3 weeks “debugging” or on some “implementation” without getting any feedback.  Chances are, you are ratholing on something that probably isn’t getting you any closer to a publication.  Seek help immediately!

• Attend every single group meeting.  Do not miss group meetings.  These are one of the most important structural elements of your graduate career that actually relates to your research.  Group meetings are important for several reasons: (1) You learn about what others in the group are doing, which may be a useful resource (or, you may find out you can be a resource to someone else).  This all helps with collaborating across the group. (2) You find out what your advisor has been up to and why he or she has not been replying to your emails immediately. (3) You can quickly identify if you need to have a longer meeting with your advisor, with other students in the group, etc.  This can be a huge timesaver.  (4) Group meetings mark the passage of time.  It is useful to hold yourself accountable and make sure that weeks and months don’t slip away without progress.  I have group meetings with my students three times a week; initially, I thought that this might be excessive, but it turns out to work pretty well.  Three short group meetings can often be a lot better than one extended group meeting.  I will expand on this more in a later post.

• If you need more of your advisor’s time, ask for it.  Students are often confused or concerned that an advisor spends more time with some students than with others and may even (wrongly) think that the advisor is either less excited about a particular project or (worse) doesn’t like some students as much as others.  (I remember comparing notes with my fellow Ph.D. students in grad school about how much time our advisor was spending with each of us.)  Yet, it is important to remember that good advisors don’t play favorites.  The time that an advisor spends with a student (or on a project) is typically determined by the advisor’s perception of how much time is needed; the required time can vary dramatically according to both the stage of the project and the stage of the student’s development.  Students who are early in their careers typically need (and should be asking for) a lot of guidance and “closed loop” feedback.  Students who are close to graduating also tend to need more attention of a different sort—help with building their professional network, seeking out job prospects, practicing job talks, and generally landing on their feet.  Similarly, nascent research projects or projects with substantial coordination components (e.g., large systems-building efforts) often need a lot of advisor attention, since they have lots of moving parts and can involve coordination between multiple sub-projects and students. Do not be overly concerned about strict time accounting.  If you feel you need more time, simply ask for it—or, better yet, just try to take more time (walk into your advisor’s office, approach him or her on IM, send regular email updates…whatever it takes).  Advisors tend to spend more time with students who demand more of their time.

• Keep your emails short and to the point.  Here is a simple rule of thumb: If the email is longer than one paragraph, it probably won’t get read right away, particularly if there is no summary at the beginning of it.  It almost certainly won’t get an immediate response.  Additionally, consider whether email is the fastest way to resolve something, or whether it’s quicker to have a 5-10 minute meeting, hangout, IM chat, phone call, or whatever.  Use the right communication mode for the job.

• Do not assume that if your email doesn’t get a reply, it hasn’t been read.  I read everything in my inbox, almost always on the same day that it arrives.  Unfortunately, I also receive 300-500 emails per day in my inbox (not mailing lists), many of which are actionable.  Suppose that half of those emails required action, and that each one required one minute to process and respond to—that’s already six or seven hours a day just to process email.  That is insane and can kill anyone’s productivity.  I am convinced that it is possible for a professor to do nothing else in life except reply to email.  To control this insanity, I often process emails “in batch mode”—leaving email to (mostly) pile up for a few days and then responding to a bunch at once.  I tell my students that if they do not receive a reply right away, “retransmission” after a few days is fine. I do not consider this to be rude, nagging, or pestering behavior; most likely I have simply just forgotten (I have found that it’s surprisingly difficult to even keep a to do list for all of these things that students ask professors to do, as doing so becomes a monster mega-task in and of itself).  Before sending a retransmission (or initial email), however, consider whether you have chosen the best medium for your message. Sometimes an in-person meeting or IM follow up to a an email will get the response you want/need.

• Make the meetings count.  Many meetings are wasted by not asking yourself simple “does this make sense?” questions before presenting a plot/result.  I ask my students to read Jon Bentley’s “Programming Pearls”, particularly the chapter on back of the envelope calculations.  Also, I advise my students to read Vern Paxson’s “Strategies for Sound Internet Measurement”. Your advisor has almost certainly seen a ton of plots/experiments/data and is pretty good at quickly determining whether a graph that you spent two days producing makes any sense at all.  You can have a more productive meeting if you do some simple debugging of plots before hand.  On this note, bringing specific, concrete things that your advisor can react to is helpful.  “I ran some experiments and things seem to look OK.” is a report I have heard many times from students.  Such a report is utterly useless.  Even if it were true (often things may not be OK), it is impossible to give feedback on or brainstorm based on vague statements.  You are likely to get a “sounds good!” in response, which is equally useless for you.  Bring something concrete to discuss.  You can present anything: A performance number, a paragraph of writing, a plot, … something to react to and figure out next steps.  Even a plot that appears buggy or inexplicable is sometimes a good topic for a meeting, too, presuming you’ve recognized the discrepancies and can’t figure out the problem.  Sometimes what appears to be a bug might in fact be an interesting artifact, or even the spark for a new paper or discovery.

• Take notes and organize them. The students who make the best use of meetings tend to have: (1) an agenda beforehand; (2) minutes afterwards; (3) something focused and concrete to discuss/think about/talk about; (4) a consolidated place to keep minutes.  Your advisor can read these minutes to prepare for the upcoming meeting, think about problems offline, review/think about the problem outside of meetings, and guide progress.  Sometimes your advisor may take notes, sometimes not.  Don’t count on it.  Even if your advisor is taking notes, your notes will complement and fill in gaps.  Different people remember different things.  Taking notes is also an important opportunity to practice writing—and students need to practice writing at every opportunity (more on that in a later post).

• Do not wait until the last minute to write your paper.  Most graduate students are working on one or at most two papers or projects at any given time.  It can thus be easy to overlook the fact  that your advisor is involved in many more things (albeit at a higher level) and, from a purely practical standpoint, might be submitting two or three papers to the same conference deadline. Thus, waiting until the last minute to write a paper draft (or complete a project) is an invitation for scattered, distracted, and superficial feedback (and severely diminished chances of a strong paper submission).   Can you write a good paper or think clearly while doing four things at once?  If not, consider your poor advisor, whose aging brain is no longer as agile as yours.  Write early, write often.  Writing is not a task that happens after the research is done; rather, it is part of the research and thinking process, not something that is done when the research is done.  Writing is part of the research.  I ask my students to have a complete paper draft at least one week before the deadline.  Nobody ever follows this advice, and I think that we can recognize that it is idealistic.  I’ve periodically threatened to ban paper submissions if there is no draft a week before; I don’t have the will to do that, although I know at least one of my collaborators who enforces this rule.  Still, the point remains: early attention == focused attention == good attention.

• Do not ask for a recommendation letter with less than one week’s notice.  A letter takes at least an hour to write—longer if there is no earlier draft from another instance.  Short notice makes for letters that will probably not be as strong as they could be, because a good letter takes time to polish.  Consider writing the first draft yourself, or at least putting some points into bullet form or providing an up-to-date CV, for quick reference.  All of this stuff makes the letter stronger and easier to write.

This list is mostly based on tips and tricks that I have found work for me.  I refined this list of advice after a discussion with Professor Jennifer Rexford, who is also full of useful advice.  Jen’s advice for new graduate students is particularly useful; I am in strong agreement with her thoughts on the benefits of regularly coming to the lab and integrating with a research group.  I’m interested to hear what other tips and tricks people have for managing their advisor(s), or thoughts from other faculty members about tips they find that work well.  In the coming weeks and months, I will follow up with specific posts on advice for writing, preparing talks, and managing time.

Nominations for the annual SIGCOMM Award are due on March 31st to Ramesh Govindan ([email protected]). The nomination procedure is described here.

The SIGCOMM Education mailing list has been created to allow all people interested in networking education to exchange information. The list is hosted on ACM's servers and you can subscribe to the list through the following web interface : http://listserv.acm.org/scripts/wa.exe?A0=sigcomm-edu-dir