Skip to content
This repository was archived by the owner on Aug 15, 2022. It is now read-only.

Machine mount sync package - initial version - #10332

Merged
cihangir merged 4 commits into
masterfrom
machine_mount_sync_init
Jan 20, 2017
Merged

Machine mount sync package - initial version#10332
cihangir merged 4 commits into
masterfrom
machine_mount_sync_init

Conversation

@ppknap

@ppknap ppknap commented Jan 17, 2017

Copy link
Copy Markdown
Contributor

Depends on: #10306

Code coverage: 81.8% of statements

Architecture

image

  • Sync - type responsible for syncing remote and local indexes.

How Has This Been Tested?

Unit tests.

Screenshots (if appropriate):

none

Types of changes

  • New feature (non-breaking change which adds functionality)

// SaveIndex atomically saves the provided index under a given path.
func SaveIndex(idx *Index, path string) (err error) {
dir, name := filepath.Split(path)
f, err := ioutil.TempFile(dir, name+"_")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need name+"_", the file is going to be suffixed either way?

f, err := ioutil.TempFile(filepath.Split(path))

@rjeczalik

Copy link
Copy Markdown
Contributor

@ppknap Please rebase.

if opts.Log != nil {
s.log = opts.Log.New("sync")
} else {
s.log = machine.DefaultLogger.New("sync")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be a global var.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And assigned at L96.

@ppknap ppknap Jan 19, 2017

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry but I can't see any advantages of this approach. However, I see one big disadvantage which is an introduction of global variable used only in one place as a fallback to incomplete options(used in tests) inside the type that is meant to be used as singleton.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not quite follow, since you're using a global variable either way (machine.DefaultLogger) - I just commented to do the same here - use a global variable instead of allocating new default logger each time.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The root logger machine.DefaultLogger is a parent for other loggers and it uses config.Konfig.Debug variable during its initialization.

Actually, I'm avoiding unnecessary allocation by not defining global variable. Please note, that this branch is a fallback which normally will not be executed.

sd, ok := s.syncs[id]
s.mu.RUnlock()

if ok {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure these fine-grained locks are correct. Two different threads can simultaneously pass this check and both can assign a new *synced value with the same id to the map. While if the whole body was protected with a single lock, the second thread would fail here.

@ppknap ppknap Jan 19, 2017

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the funny fact is that we don't care.. even if IDs are the same (which is not possible in valid logic), the only inconsistency will be that we fetch index twice (or just load it from cache) and swap the file atomically.

This is a trade off I had to make if I want to add mounts concurrently (during kd initialization). newSync may be slow and I want to avoid synchronous initialization process.

I will add:

s.mu.Lock()
if _, ok := s.syncs[id]; ok {
    s.mu.Unlock()
    return //some fancy error here
}
s.syncs[id] = sd
s.mu.Unlock()

@ppknap

ppknap commented Jan 19, 2017

Copy link
Copy Markdown
Contributor Author

@rjeczalik done

// mktree ensures that synced working directory is created.
func (s *synced) mktree() error {
dataPath := filepath.Join(s.wd, "data")
info, err := os.Stat(dataPath)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prob MkdirAll is enough, it already handles Stat as fast path https://golang.org/src/os/path.go?s=537:766

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - TIL

@cihangir
cihangir merged commit 58337f7 into master Jan 20, 2017
@cihangir
cihangir deleted the machine_mount_sync_init branch January 20, 2017 14:17
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants