mirror of
https://github.com/opencloud-eu/opencloud.git
synced 2026-01-06 04:09:40 -06:00
switch to go vendoring
This commit is contained in:
1
vendor/github.com/orcaman/concurrent-map/.gitignore
generated
vendored
Normal file
1
vendor/github.com/orcaman/concurrent-map/.gitignore
generated
vendored
Normal file
@@ -0,0 +1 @@
|
||||
go.mod
|
||||
32
vendor/github.com/orcaman/concurrent-map/.travis.yml
generated
vendored
Normal file
32
vendor/github.com/orcaman/concurrent-map/.travis.yml
generated
vendored
Normal file
@@ -0,0 +1,32 @@
|
||||
# This is a weird way of telling Travis to use the fast container-based test
|
||||
# runner instead of the slow VM-based runner.
|
||||
sudo: false
|
||||
|
||||
language: go
|
||||
|
||||
# You don't need to test on very old version of the Go compiler. It's the user's
|
||||
# responsibility to keep their compilers up to date.
|
||||
go:
|
||||
- 1.12.x
|
||||
|
||||
# Only clone the most recent commit.
|
||||
git:
|
||||
depth: 1
|
||||
|
||||
# Skip the install step. Don't `go get` dependencies. Only build with the code
|
||||
# in vendor/
|
||||
install: true
|
||||
|
||||
# Don't email me the results of the test runs.
|
||||
notifications:
|
||||
email: false
|
||||
|
||||
before_script:
|
||||
- curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh | sh -s -- -b $(go env GOPATH)/bin latest
|
||||
|
||||
# script always runs to completion (set +e). If we have linter issues AND a
|
||||
# failing test, we want to see both. Configure golangci-lint with a
|
||||
# .golangci.yml file at the top level of your repo.
|
||||
script:
|
||||
- golangci-lint run # run a bunch of code checkers/linters in parallel
|
||||
- go test -v -race ./... # Run all the tests with the race detector enabled
|
||||
22
vendor/github.com/orcaman/concurrent-map/LICENSE
generated
vendored
Normal file
22
vendor/github.com/orcaman/concurrent-map/LICENSE
generated
vendored
Normal file
@@ -0,0 +1,22 @@
|
||||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2014 streamrail
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
64
vendor/github.com/orcaman/concurrent-map/README-zh.md
generated
vendored
Normal file
64
vendor/github.com/orcaman/concurrent-map/README-zh.md
generated
vendored
Normal file
@@ -0,0 +1,64 @@
|
||||
# concurrent map [](https://travis-ci.com/orcaman/concurrent-map)
|
||||
|
||||
正如 [这里](http://golang.org/doc/faq#atomic_maps) 和 [这里](http://blog.golang.org/go-maps-in-action)所描述的, Go语言原生的`map`类型并不支持并发读写。`concurrent-map`提供了一种高性能的解决方案:通过对内部`map`进行分片,降低锁粒度,从而达到最少的锁等待时间(锁冲突)
|
||||
|
||||
在Go 1.9之前,go语言标准库中并没有实现并发`map`。在Go 1.9中,引入了`sync.Map`。新的`sync.Map`与此`concurrent-map`有几个关键区别。标准库中的`sync.Map`是专为`append-only`场景设计的。因此,如果您想将`Map`用于一个类似内存数据库,那么使用我们的版本可能会受益。你可以在golang repo上读到更多,[这里](https://github.com/golang/go/issues/21035) and [这里](https://stackoverflow.com/questions/11063473/map-with-concurrent-access)
|
||||
***译注:`sync.Map`在读多写少性能比较好,否则并发性能很差***
|
||||
|
||||
## 用法
|
||||
|
||||
导入包:
|
||||
|
||||
```go
|
||||
import (
|
||||
"github.com/orcaman/concurrent-map"
|
||||
)
|
||||
|
||||
```
|
||||
|
||||
```bash
|
||||
go get "github.com/orcaman/concurrent-map"
|
||||
```
|
||||
|
||||
现在包被导入到了`cmap`命名空间下
|
||||
***译注:通常包的限定前缀(命名空间)是和目录名一致的,但是这个包有点典型😂,不一致!!!所以用的时候注意***
|
||||
|
||||
## 示例
|
||||
|
||||
```go
|
||||
|
||||
// 创建一个新的 map.
|
||||
m := cmap.New()
|
||||
|
||||
// 设置变量m一个键为“foo”值为“bar”键值对
|
||||
m.Set("foo", "bar")
|
||||
|
||||
// 从m中获取指定键值.
|
||||
if tmp, ok := m.Get("foo"); ok {
|
||||
bar := tmp.(string)
|
||||
}
|
||||
|
||||
// 删除键为“foo”的项
|
||||
m.Remove("foo")
|
||||
|
||||
```
|
||||
|
||||
更多使用示例请查看`concurrent_map_test.go`.
|
||||
|
||||
运行测试:
|
||||
|
||||
```bash
|
||||
go test "github.com/orcaman/concurrent-map"
|
||||
```
|
||||
|
||||
## 贡献说明
|
||||
|
||||
我们非常欢迎大家的贡献。如欲合并贡献,请遵循以下指引:
|
||||
- 新建一个issue,并且叙述为什么这么做(解决一个bug,增加一个功能,等等)
|
||||
- 根据核心团队对上述问题的反馈,提交一个PR,描述变更并链接到该问题。
|
||||
- 新代码必须具有测试覆盖率。
|
||||
- 如果代码是关于性能问题的,则必须在流程中包括基准测试(无论是在问题中还是在PR中)。
|
||||
- 一般来说,我们希望`concurrent-map`尽可能简单,且与原生的`map`有相似的操作。当你新建issue时请注意这一点。
|
||||
|
||||
## 许可证
|
||||
MIT (see [LICENSE](https://github.com/orcaman/concurrent-map/blob/master/LICENSE) file)
|
||||
65
vendor/github.com/orcaman/concurrent-map/README.md
generated
vendored
Normal file
65
vendor/github.com/orcaman/concurrent-map/README.md
generated
vendored
Normal file
@@ -0,0 +1,65 @@
|
||||
# concurrent map [](https://travis-ci.com/orcaman/concurrent-map)
|
||||
|
||||
As explained [here](http://golang.org/doc/faq#atomic_maps) and [here](http://blog.golang.org/go-maps-in-action), the `map` type in Go doesn't support concurrent reads and writes. `concurrent-map` provides a high-performance solution to this by sharding the map with minimal time spent waiting for locks.
|
||||
|
||||
Prior to Go 1.9, there was no concurrent map implementation in the stdlib. In Go 1.9, `sync.Map` was introduced. The new `sync.Map` has a few key differences from this map. The stdlib `sync.Map` is designed for append-only scenarios. So if you want to use the map for something more like in-memory db, you might benefit from using our version. You can read more about it in the golang repo, for example [here](https://github.com/golang/go/issues/21035) and [here](https://stackoverflow.com/questions/11063473/map-with-concurrent-access)
|
||||
|
||||
## usage
|
||||
|
||||
Import the package:
|
||||
|
||||
```go
|
||||
import (
|
||||
"github.com/orcaman/concurrent-map"
|
||||
)
|
||||
|
||||
```
|
||||
|
||||
```bash
|
||||
go get "github.com/orcaman/concurrent-map"
|
||||
```
|
||||
|
||||
The package is now imported under the "cmap" namespace.
|
||||
|
||||
## example
|
||||
|
||||
```go
|
||||
|
||||
// Create a new map.
|
||||
m := cmap.New()
|
||||
|
||||
// Sets item within map, sets "bar" under key "foo"
|
||||
m.Set("foo", "bar")
|
||||
|
||||
// Retrieve item from map.
|
||||
if tmp, ok := m.Get("foo"); ok {
|
||||
bar := tmp.(string)
|
||||
}
|
||||
|
||||
// Removes item under key "foo"
|
||||
m.Remove("foo")
|
||||
|
||||
```
|
||||
|
||||
For more examples have a look at concurrent_map_test.go.
|
||||
|
||||
Running tests:
|
||||
|
||||
```bash
|
||||
go test "github.com/orcaman/concurrent-map"
|
||||
```
|
||||
|
||||
## guidelines for contributing
|
||||
|
||||
Contributions are highly welcome. In order for a contribution to be merged, please follow these guidelines:
|
||||
- Open an issue and describe what you are after (fixing a bug, adding an enhancement, etc.).
|
||||
- According to the core team's feedback on the above mentioned issue, submit a pull request, describing the changes and linking to the issue.
|
||||
- New code must have test coverage.
|
||||
- If the code is about performance issues, you must include benchmarks in the process (either in the issue or in the PR).
|
||||
- In general, we would like to keep `concurrent-map` as simple as possible and as similar to the native `map`. Please keep this in mind when opening issues.
|
||||
|
||||
## language
|
||||
- [中文说明](./README-zh.md)
|
||||
|
||||
## license
|
||||
MIT (see [LICENSE](https://github.com/orcaman/concurrent-map/blob/master/LICENSE) file)
|
||||
343
vendor/github.com/orcaman/concurrent-map/concurrent_map.go
generated
vendored
Normal file
343
vendor/github.com/orcaman/concurrent-map/concurrent_map.go
generated
vendored
Normal file
@@ -0,0 +1,343 @@
|
||||
package cmap
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"sync"
|
||||
)
|
||||
|
||||
var SHARD_COUNT = 32
|
||||
|
||||
// A "thread" safe map of type string:Anything.
|
||||
// To avoid lock bottlenecks this map is dived to several (SHARD_COUNT) map shards.
|
||||
type ConcurrentMap []*ConcurrentMapShared
|
||||
|
||||
// A "thread" safe string to anything map.
|
||||
type ConcurrentMapShared struct {
|
||||
items map[string]interface{}
|
||||
sync.RWMutex // Read Write mutex, guards access to internal map.
|
||||
}
|
||||
|
||||
// Creates a new concurrent map.
|
||||
func New() ConcurrentMap {
|
||||
m := make(ConcurrentMap, SHARD_COUNT)
|
||||
for i := 0; i < SHARD_COUNT; i++ {
|
||||
m[i] = &ConcurrentMapShared{items: make(map[string]interface{})}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// GetShard returns shard under given key
|
||||
func (m ConcurrentMap) GetShard(key string) *ConcurrentMapShared {
|
||||
return m[uint(fnv32(key))%uint(SHARD_COUNT)]
|
||||
}
|
||||
|
||||
func (m ConcurrentMap) MSet(data map[string]interface{}) {
|
||||
for key, value := range data {
|
||||
shard := m.GetShard(key)
|
||||
shard.Lock()
|
||||
shard.items[key] = value
|
||||
shard.Unlock()
|
||||
}
|
||||
}
|
||||
|
||||
// Sets the given value under the specified key.
|
||||
func (m ConcurrentMap) Set(key string, value interface{}) {
|
||||
// Get map shard.
|
||||
shard := m.GetShard(key)
|
||||
shard.Lock()
|
||||
shard.items[key] = value
|
||||
shard.Unlock()
|
||||
}
|
||||
|
||||
// Callback to return new element to be inserted into the map
|
||||
// It is called while lock is held, therefore it MUST NOT
|
||||
// try to access other keys in same map, as it can lead to deadlock since
|
||||
// Go sync.RWLock is not reentrant
|
||||
type UpsertCb func(exist bool, valueInMap interface{}, newValue interface{}) interface{}
|
||||
|
||||
// Insert or Update - updates existing element or inserts a new one using UpsertCb
|
||||
func (m ConcurrentMap) Upsert(key string, value interface{}, cb UpsertCb) (res interface{}) {
|
||||
shard := m.GetShard(key)
|
||||
shard.Lock()
|
||||
v, ok := shard.items[key]
|
||||
res = cb(ok, v, value)
|
||||
shard.items[key] = res
|
||||
shard.Unlock()
|
||||
return res
|
||||
}
|
||||
|
||||
// Sets the given value under the specified key if no value was associated with it.
|
||||
func (m ConcurrentMap) SetIfAbsent(key string, value interface{}) bool {
|
||||
// Get map shard.
|
||||
shard := m.GetShard(key)
|
||||
shard.Lock()
|
||||
_, ok := shard.items[key]
|
||||
if !ok {
|
||||
shard.items[key] = value
|
||||
}
|
||||
shard.Unlock()
|
||||
return !ok
|
||||
}
|
||||
|
||||
// Get retrieves an element from map under given key.
|
||||
func (m ConcurrentMap) Get(key string) (interface{}, bool) {
|
||||
// Get shard
|
||||
shard := m.GetShard(key)
|
||||
shard.RLock()
|
||||
// Get item from shard.
|
||||
val, ok := shard.items[key]
|
||||
shard.RUnlock()
|
||||
return val, ok
|
||||
}
|
||||
|
||||
// Count returns the number of elements within the map.
|
||||
func (m ConcurrentMap) Count() int {
|
||||
count := 0
|
||||
for i := 0; i < SHARD_COUNT; i++ {
|
||||
shard := m[i]
|
||||
shard.RLock()
|
||||
count += len(shard.items)
|
||||
shard.RUnlock()
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
// Looks up an item under specified key
|
||||
func (m ConcurrentMap) Has(key string) bool {
|
||||
// Get shard
|
||||
shard := m.GetShard(key)
|
||||
shard.RLock()
|
||||
// See if element is within shard.
|
||||
_, ok := shard.items[key]
|
||||
shard.RUnlock()
|
||||
return ok
|
||||
}
|
||||
|
||||
// Remove removes an element from the map.
|
||||
func (m ConcurrentMap) Remove(key string) {
|
||||
// Try to get shard.
|
||||
shard := m.GetShard(key)
|
||||
shard.Lock()
|
||||
delete(shard.items, key)
|
||||
shard.Unlock()
|
||||
}
|
||||
|
||||
// RemoveCb is a callback executed in a map.RemoveCb() call, while Lock is held
|
||||
// If returns true, the element will be removed from the map
|
||||
type RemoveCb func(key string, v interface{}, exists bool) bool
|
||||
|
||||
// RemoveCb locks the shard containing the key, retrieves its current value and calls the callback with those params
|
||||
// If callback returns true and element exists, it will remove it from the map
|
||||
// Returns the value returned by the callback (even if element was not present in the map)
|
||||
func (m ConcurrentMap) RemoveCb(key string, cb RemoveCb) bool {
|
||||
// Try to get shard.
|
||||
shard := m.GetShard(key)
|
||||
shard.Lock()
|
||||
v, ok := shard.items[key]
|
||||
remove := cb(key, v, ok)
|
||||
if remove && ok {
|
||||
delete(shard.items, key)
|
||||
}
|
||||
shard.Unlock()
|
||||
return remove
|
||||
}
|
||||
|
||||
// Pop removes an element from the map and returns it
|
||||
func (m ConcurrentMap) Pop(key string) (v interface{}, exists bool) {
|
||||
// Try to get shard.
|
||||
shard := m.GetShard(key)
|
||||
shard.Lock()
|
||||
v, exists = shard.items[key]
|
||||
delete(shard.items, key)
|
||||
shard.Unlock()
|
||||
return v, exists
|
||||
}
|
||||
|
||||
// IsEmpty checks if map is empty.
|
||||
func (m ConcurrentMap) IsEmpty() bool {
|
||||
return m.Count() == 0
|
||||
}
|
||||
|
||||
// Used by the Iter & IterBuffered functions to wrap two variables together over a channel,
|
||||
type Tuple struct {
|
||||
Key string
|
||||
Val interface{}
|
||||
}
|
||||
|
||||
// Iter returns an iterator which could be used in a for range loop.
|
||||
//
|
||||
// Deprecated: using IterBuffered() will get a better performence
|
||||
func (m ConcurrentMap) Iter() <-chan Tuple {
|
||||
chans := snapshot(m)
|
||||
ch := make(chan Tuple)
|
||||
go fanIn(chans, ch)
|
||||
return ch
|
||||
}
|
||||
|
||||
// IterBuffered returns a buffered iterator which could be used in a for range loop.
|
||||
func (m ConcurrentMap) IterBuffered() <-chan Tuple {
|
||||
chans := snapshot(m)
|
||||
total := 0
|
||||
for _, c := range chans {
|
||||
total += cap(c)
|
||||
}
|
||||
ch := make(chan Tuple, total)
|
||||
go fanIn(chans, ch)
|
||||
return ch
|
||||
}
|
||||
|
||||
// Clear removes all items from map.
|
||||
func (m ConcurrentMap) Clear() {
|
||||
for item := range m.IterBuffered() {
|
||||
m.Remove(item.Key)
|
||||
}
|
||||
}
|
||||
|
||||
// Returns a array of channels that contains elements in each shard,
|
||||
// which likely takes a snapshot of `m`.
|
||||
// It returns once the size of each buffered channel is determined,
|
||||
// before all the channels are populated using goroutines.
|
||||
func snapshot(m ConcurrentMap) (chans []chan Tuple) {
|
||||
chans = make([]chan Tuple, SHARD_COUNT)
|
||||
wg := sync.WaitGroup{}
|
||||
wg.Add(SHARD_COUNT)
|
||||
// Foreach shard.
|
||||
for index, shard := range m {
|
||||
go func(index int, shard *ConcurrentMapShared) {
|
||||
// Foreach key, value pair.
|
||||
shard.RLock()
|
||||
chans[index] = make(chan Tuple, len(shard.items))
|
||||
wg.Done()
|
||||
for key, val := range shard.items {
|
||||
chans[index] <- Tuple{key, val}
|
||||
}
|
||||
shard.RUnlock()
|
||||
close(chans[index])
|
||||
}(index, shard)
|
||||
}
|
||||
wg.Wait()
|
||||
return chans
|
||||
}
|
||||
|
||||
// fanIn reads elements from channels `chans` into channel `out`
|
||||
func fanIn(chans []chan Tuple, out chan Tuple) {
|
||||
wg := sync.WaitGroup{}
|
||||
wg.Add(len(chans))
|
||||
for _, ch := range chans {
|
||||
go func(ch chan Tuple) {
|
||||
for t := range ch {
|
||||
out <- t
|
||||
}
|
||||
wg.Done()
|
||||
}(ch)
|
||||
}
|
||||
wg.Wait()
|
||||
close(out)
|
||||
}
|
||||
|
||||
// Items returns all items as map[string]interface{}
|
||||
func (m ConcurrentMap) Items() map[string]interface{} {
|
||||
tmp := make(map[string]interface{})
|
||||
|
||||
// Insert items to temporary map.
|
||||
for item := range m.IterBuffered() {
|
||||
tmp[item.Key] = item.Val
|
||||
}
|
||||
|
||||
return tmp
|
||||
}
|
||||
|
||||
// Iterator callback,called for every key,value found in
|
||||
// maps. RLock is held for all calls for a given shard
|
||||
// therefore callback sess consistent view of a shard,
|
||||
// but not across the shards
|
||||
type IterCb func(key string, v interface{})
|
||||
|
||||
// Callback based iterator, cheapest way to read
|
||||
// all elements in a map.
|
||||
func (m ConcurrentMap) IterCb(fn IterCb) {
|
||||
for idx := range m {
|
||||
shard := (m)[idx]
|
||||
shard.RLock()
|
||||
for key, value := range shard.items {
|
||||
fn(key, value)
|
||||
}
|
||||
shard.RUnlock()
|
||||
}
|
||||
}
|
||||
|
||||
// Keys returns all keys as []string
|
||||
func (m ConcurrentMap) Keys() []string {
|
||||
count := m.Count()
|
||||
ch := make(chan string, count)
|
||||
go func() {
|
||||
// Foreach shard.
|
||||
wg := sync.WaitGroup{}
|
||||
wg.Add(SHARD_COUNT)
|
||||
for _, shard := range m {
|
||||
go func(shard *ConcurrentMapShared) {
|
||||
// Foreach key, value pair.
|
||||
shard.RLock()
|
||||
for key := range shard.items {
|
||||
ch <- key
|
||||
}
|
||||
shard.RUnlock()
|
||||
wg.Done()
|
||||
}(shard)
|
||||
}
|
||||
wg.Wait()
|
||||
close(ch)
|
||||
}()
|
||||
|
||||
// Generate keys
|
||||
keys := make([]string, 0, count)
|
||||
for k := range ch {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
return keys
|
||||
}
|
||||
|
||||
//Reviles ConcurrentMap "private" variables to json marshal.
|
||||
func (m ConcurrentMap) MarshalJSON() ([]byte, error) {
|
||||
// Create a temporary map, which will hold all item spread across shards.
|
||||
tmp := make(map[string]interface{})
|
||||
|
||||
// Insert items to temporary map.
|
||||
for item := range m.IterBuffered() {
|
||||
tmp[item.Key] = item.Val
|
||||
}
|
||||
return json.Marshal(tmp)
|
||||
}
|
||||
|
||||
func fnv32(key string) uint32 {
|
||||
hash := uint32(2166136261)
|
||||
const prime32 = uint32(16777619)
|
||||
keyLength := len(key)
|
||||
for i := 0; i < keyLength; i++ {
|
||||
hash *= prime32
|
||||
hash ^= uint32(key[i])
|
||||
}
|
||||
return hash
|
||||
}
|
||||
|
||||
// Concurrent map uses Interface{} as its value, therefor JSON Unmarshal
|
||||
// will probably won't know which to type to unmarshal into, in such case
|
||||
// we'll end up with a value of type map[string]interface{}, In most cases this isn't
|
||||
// out value type, this is why we've decided to remove this functionality.
|
||||
|
||||
// func (m *ConcurrentMap) UnmarshalJSON(b []byte) (err error) {
|
||||
// // Reverse process of Marshal.
|
||||
|
||||
// tmp := make(map[string]interface{})
|
||||
|
||||
// // Unmarshal into a single map.
|
||||
// if err := json.Unmarshal(b, &tmp); err != nil {
|
||||
// return nil
|
||||
// }
|
||||
|
||||
// // foreach key,value pair in temporary map insert into our concurrent map.
|
||||
// for key, val := range tmp {
|
||||
// m.Set(key, val)
|
||||
// }
|
||||
// return nil
|
||||
// }
|
||||
Reference in New Issue
Block a user