Skip to content

Comment on Porting dl.google.com from C++ to Go

Comments

While I won't dispute that Go has some cute primitives, I thought the examples were terrible. On slide 25, it talks about why a simple operation is painful (http://talks.golang.org/2013/oscon-dl.slide#25), and then goes on to evangelize io.Copy() on slide 31. Okay, so the standard library saves me from open-coding it:

  func Copy(dst Writer, src Reader) (written int64, err error) {
      // If the reader has a WriteTo method, use it to do the copy.
      // Avoids an allocation and a copy.
      if wt, ok := src.(WriterTo); ok {
          return wt.WriteTo(dst)
      }
      // Similarly, if the writer has a ReadFrom method, use it to do the copy.
      if rt, ok := dst.(ReaderFrom); ok {
          return rt.ReadFrom(src)
      }
      buf := make([]byte, 32*1024)
      for {
          nr, er := src.Read(buf)
          if nr > 0 {
              nw, ew := dst.Write(buf[0:nr])
              if nw > 0 {
                  written += int64(nw)
              }
              if ew != nil {
                  err = ew
                  break
              }
              if nr != nw {
                  err = ErrShortWrite
                  break
              }
          }
          if er == EOF {
              break
          }
          if er != nil {
              err = er
              break
          }
      }
      return written, err
  }
Uh, big deal?

The chunk of what's important isn't explained at all:

- runtime/ takes care of memory management quite efficiently with a decent tracing gc in runtime/mgc0.c. I haven't benchmarked it against other stop-the-world collectors, but it should be no match for truly concurrent gc.

- runtime/proc.c schedules various blocking and non-blocking (called netpoll, which resolves to epoll on systems where it is available) calls. It seems to account for number of cores and use native threads, but I'm not sure how it interacts with the Linux scheduler.

- runtime/malloc.goc is the core memory allocator/deallocator. Seems to be a relatively straighforward arena allocator using a bitmap.

I didn't have time to go through groupcache, but the presentation certainly didn't tell me much about it.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.