Hi, /u/themurmel!
>How are you deploying Kubernetes?
We're using Packer + Terraform + kubeadm and a sprinkling of Puppet.
>What tools are you using for CI/CD?
Drone for CI, Spinnaker for CD.
>How are you managing authentication/authorization to Kubernetes?
We're using OpenID Connect with Okta as our IDP, using the groups in the JWT for RBAC. Hm, I only managed to fit a few acronyms in there...
We're about to start poking with Open Policy Agent, as well!
>Anything you would like to change compared to how it is today?
I'd love to see deeper or more seamless Kubernetes support for Vault.
Drone.io!
Would give you all the good stuff on circle, but for free and much faster. Try it our.
I even wrote a post about my own experience and usage if you're interested -
https://medium.com/prodopsio/how-i-helped-my-company-ship-features-10-times-faster-and-made-dev-and-ops-win-a758a83b530c
BTW. If you were testing with build 250 it is recommended you download a new dev build from (251): https://drone.io/github.com/McJty/RFTools
In 250 I accidently made the mistake of loading every dimension to make the entities freeze. That makes everything very slow. 251 only freezes entities on unpowered and actually loaded dimensions.
“people learn and use Jenkins because people already use Jenkins”
Accurate in my experience. Like Python for Machine Learning, it's the accrued the most venerable library of tools for the domain. That does not equate to good, but it can equate to necessary for many, many teams.
Since we like to containerize everything that isn't serverless, including all our tools, I've had success porting things to Drone. I much prefer writing standardized tools to do all our CI/CD over the sprawl of a hundred Jenkinsfiles constantly accruing idiom drift.
If you are looking to replace Jenkins, I would recommend Drone. It's container based, which means rolling your own plugins is very easy, and has upcoming support for secrets from Kubernetes and Hashicorp Vault.
Well, Gitea doesn't currently have CI as far as I know, so you'd need to have something to build your site after it's updated. Personally I use Drone CI for this, which is a decently lightweight solution.
After that its pretty much just a matter of some configuration. Have Gitea send a webhook to Drone when a new commit happens, have Drone build it with Hugo, then copy it to a specific directory on your server. After that point Caddy to it, and you should be done.
Jenkins is problematic inherently; not only it's designed as a hard to backup system, based on XMLs and requires hacky backups, it tends to break and slow down.
Yet, I can handle all of the above, my problem is the very wide range of custom pipelines it provides. In first sight it may be attractive, but Jenkins tends to be abused and turn in to a bash scripting schedule engine which is hard to track, debug or migrate if needed.
Having configuration as code is optional, but it requires using a non conventional DSL.
If you think about using that or circle, I'd choose circle any day (or Travis for that matter, and my personal favorite - drone.io).
I know that 95% of the world uses Jenkins, but I'd go with a system that enforces configuration as code, uses stateless instances and an external DB, fast and agile.
And for the love of god - a system that allows you to develop your own plugins or contribute code without doing groovy....
You don't have to wait. In fact since you asked for this feature in the first place I would appreciate it if you could test this with the latest dev build that you can get from: https://drone.io/github.com/McJty/RFTools/files
Note that the feature is disabled by default. You need to set the new freezeUnpoweredDimension option to true to enable it.
It is, it's buried in the newest Reliquary thread. I'm on a phone and can't easily paste the link, but if you Google it you can probably find it. Supposedly it's linked on the github somewhere but I could not find it there.
Edit: found it https://drone.io/github.com/TrainerGuy22/Reliquary/files
There's Drone CI which is a bit simpler than Jenkins and gaining popularity. It supports running scripts directly on agent machines, SSH-ing from agents to other machines and multiple variations on containerized agents (Docker, Kubernetes, ...)
I use it for some personal projects and am in general quite happy although it does have some quirks.
Totally with you on this one, my startup has its own small data centre with an OpenStack cluster on it, we have backup plans, emergency scenarios, all the stuff needed. We use cloud resources but in the end everything central to the business is on owned hardware. If something goes down in the cloud, we flip a switch, go to another provider and shrug it off. Most of that by now is automated using drone.io as a CI.
I get that people want to run a lean startup, and often skip the low level things. But personally I am sleeping so much better every day because I know I am the one in control and not a random cloud provider.
As for pricing, hardware is an investment and in the long run there is no way for any cloud provider to beat this. Effectively the monthly costs for running the same hardware in the cloud would be almost 10 times as high compared to the cloud pricing. Minus the full control over things this is a bad state IMHO. Not everyone can do the upfront investment though.
Yes, it is a normal inventory so a storage bus will work.
You can get the latest dev build from here: https://drone.io/github.com/McJty/RFTools/files
But be careful. I'm making lots of changes and things are not always compatible :-)
Also the things you see above are not yet there. I will commit those in less then an hour.
you pass it in as a volume -v /var/run/docker.sock:/var/run/docker.sock:ro
You will also need to install docker on the container that jenkins is using to do the build or use a prebuilt container like docker/dind or docker/compose
but I second u/mdedonno comment that you should look at using Kaniko or something similar. Personally I would also recommend evaluating drone.io or another more modern and streamlined CI system, IMHO jenkins is a pile of bailing wire and duct tape.
Other people have said it already: if you map the docker socket (usually /var/run/docker.sock
) or expose it via HTTP(s) from the host, you can docker run
inside a container and start another container on the host.
Another option is to "docker in docker" (or DIND), but that's generally frowned upon.
In the end, unless your goal is to build a CI server, you could run drone.io or Gitlab and let it start containers for you. Both have a well-defined configuration which allows you to create "pipelines" for orchestration (e.g. build step, then lint, then test) or even conditionals. Generally, I think most CI systems these days are capable to do exactly that for you.
Building a little orchestrator is still a neat task.
I've been using Drone.io on k8s for 2 years, it did what we wanted (we also wrote a bunch of drone plugins in bash/python and golang, it's great)
I recently joined a completely new startup and I want to use Spinnaker.io - I'm no longer sure if I want to set up Drone for CI here...
There is a new version of Drone (kubeci) that uses k8s jobs instead of the built-in Drone job scheduler and it is exciting (but it is very early alpha)
for now we are using keel.sh as a quick fix (bitbucket pipelines build images, keel watches registry image manifest for changes and triggers a deployment rolling update)
I have looked into:
- Brigade
- AWS Codebuild
- argo-ci
- concourse
- Jenkins
and I'm still not sure if I should choose Concourse or Jenkins... I was leaning towards Jenkins (I never used it), but this thread has me back leaning over towards concourse
You also might want to look into Drone. They have a stripped down CI environment designed around doing things with Docker. I've been following it for a while, but had issues getting it to build my project correctly.
I was wondering on how to setup drone build for a github project; on the same lines of travis. say if i have https://github.com/harrydevnull/redis-rs/blob/master/.travis.yml on my github project and is configured on travis; it would build automatically on travis.
i was trying to emulate the same behavior on https://drone.io/
Jenkins is solid, but complicated. It's one of those "kitchen sink" projects that does nothing simply, but does anything with enough work. ("XML is like violence. If it's not working, you're not using enough!")
Personally, I've been using Drone. It's a hosted service, but their code is open so you can run your own instance. They are in the middle of a re-write, so the docs for their open source version are in flux (read: nonexistent). Once I figured it out, I got it building projects from Github just by adding creds to a 3 line config file. There is only one sqlite file to backup. The interface is super-simple with no extra fluff.
It requires that you test in Docker containers. This is actually a good thing.
1) It forces you to specify all of your dependencies in your source code repo. No more "oops, I forgot I added a dependency on library X" or "Didn't you get the memo? we're running PHP 5.6 in production now".
2) New developers can start hacking in minutes, no matter what OS they use. No more spending the day trying to download years-old versions of each of your dependencies.
There is the LNP installer found here: https://github.com/andrewd18/df-lnp-installer
Which automatically downloads the Linux version of the starter pack from: https://drone.io/bitbucket.org/Dricus/lazy-newbpack which hasn't been updated for months now it seems
My company is using mainly teamcity but I don't work with it myself. However, I much prefer Gitlab CI from the limited experience I have with teamcity.
In my free time I use drone.io which works well in combination with gitea. That's the stack I would probably use it work as well for a small team without any budget (I'm tired of trying to convince people why a certain license tier would be beneficial) since I prefer KISS software in general but of course my colleagues/management disagree.
Mach doch ein Dokuwiki draus.
Kannst mich auch gerne kontaktieren und ich hoste das kostenlos. Wollte eh mal wieder drone.io aufsetzen, das wäre die ideale Gelegenheit.
Würde was dagegen sprechen, das in einem schon vorhandenen Wiki zu veröffentlichen?
>was sind die besten [VPNs] für Android
Wireguard natürlich.
Currently I'm hosting not that much.
My Main Server is a Proxmox Host with an Xeon E3-1220v5 and 64 GB RAM.
VMs:
- Home Assistant (hassio supervisor)
- Papermerge (docker)
- Docker Server
- M365 Dev DC
- M365 Dev Win10 PC
LXC:
- MariaDB Server
- ICINGA2 Monitoring
Docker:
- Plex with Tautulli
- Gitea
- Unifi Controller
- Teslamate (Tesla Data Aggregation)
- drone.io
- watchtower
- Portainer
- traefik
On my on vps (hosted by netcup)
- Traefik
- several Wordpress blogs
- Matomo
- Plausible
- Nextcloud
Second VPS:
- Email Server vor some smaller projects (main address hosted by Office 365)
There are many SaaS solutions for CI/CD pipelines (I've had good experience with drone.io ). Setting up Jenkins and maintaining it is quite the operational cost.
If that is not an option, Gitlab CI has always been my goto option before going full blown CI server.
As /u/whyitno-work already explained, the main issue is that of trust.
Technically, it can certainly be done, without much trouble. A good example is running DroneCI and having it authenticate through OAuth against a selfhosted Gitea instance. But that only works because Drone's code explicitly trusts this.
Facebook or Gmail do not provide you with an option to add this custom level of trust.
Another authentication method used by Authelia and Keycloak, is forward authentication proxy
, which most reverse proxies support:
Before any incoming request is allowed to be forwarded to the respective webserver, the reverse proxy sends a HTTP request to the forward authentication proxy, which in turn responds with either code 20x
(authentication success) or 40x
(authentication failure). The reverse proxy determines access based on that response-code.
Again, this setup relies on control over all relevant services, and again Facebook or Gmail do not provide that custom control.
I found links in your comment that were not hyperlinked:
I did the honors for you.
^delete ^| ^information ^| ^<3
First of all, thank you for the response. I will look into an alternative for the Docker runner if I find myself doing Docker builds :)
But unfortunately, maybe I am not seeing things correctly in regards to your comment about being limited for builds. I'm currently looking at your first link with the differences between OSS and Enterprise Drone. Under the usage section for the OSS version, I am seeing unlimited builds and unlimited build minutes. And looking at this page, it says the same.
Yeah, I was looking at adding something that covers continuous development. Like you said, you would need to run something like Jenkins. I was looking at using https://drone.io, it seems like it’s a lot lighter weight than Jenkins if you self host.
I don't like Jenkins. IMHO pipeline as code is required and the Jenkinsfile (pipeline as code for Jenkins) has groovy syntax :( If you can host your own CICD tool I recommend you Drone CI: https://drone.io/
I don't like Jenkins because I want to use pipeline as code and Jenkinsfile you need groovy code :( If you can host your own CICD tool you should take a look on Drone CI: https://drone.io/
Klassiker wäre Bugzilla. Aber das finde ich hässlich, hat Altlasten, etc.
Dann gab es trac. Aber weiß nicht ob es das noch gibt.
Redmine wurde bereits erwähnt.
Man kann sich sogar ein komplettes gitlab mit allem installieren. Aber das hat meinen kleinen Homeserver massiv überfordert.
Ich nutze inzwischen gitea, das ist quasi "abgespecktes github", und recht ressourcenschonend. Da gehören ja auch recht einfach gehaltene Tickets dazu. Falls du auch git nutzt, wäre das vielleicht was. Dazu passt drone.io als CI/CD-System, wenn du sowas brauchst/willst. Läuft bei mir beides in docker, die bieten gute und stets aktuelle images an.
Alles genannte ist open source, zumindest als community edition.
I would recommend just doing a `docker run` of your image on the VM, Kubernetes is cool but it makes the project way more complicated (although no shame in exploring some options) and it doesn't seem like there's a ton of services you need to run.
For CI/CD, I'd recommend something like Drone https://drone.io/. It's free and super simple. You can use it to automate the image build and calling some script on the VM to pull new images and restart the container.
Just an FYI as most people somehow miss this, Drone with Kubernetes runners requires the enterprise version which has a license with a cost unless you are a small business
https://drone.io/enterprise/features/
https://docs.drone.io/enterprise/#is-drone-enterprise-free-for-small-businesses
Ok double checked docs and if you use K8s runners you need the enterprise version, and this is what I originally meant with k8s support (I.e. not just the master)... maybe too implicit :)
I just use CI/CD. Once setup, it's pretty easy to create more projects unless your configuration is drastically different.
Bitbucket, Gitlab, Github, Drone.io and CircleCI all offer free or low cost services. I switched to CircleCI last week because it's independent and (as far as I can I tell) be easily switched around to different providers.
We use Drone for CI, and Spinnaker for k8s deployments. We host both of these ourselves. Non-k8s deployments are handled through an in-house tool, Rollingpin.
For group project, framework open to what the team decides. Technically you can you any stack for your project, and depend on the topic of your project (it could be a mobile or web app). Most of the project is web-based. My team used Angular for front-end and Java servlet to implement Rest API endpoints. So decide what stack to used is not that hard.
The hard part is the deployment process. For last semester, we used Dockers, Drone.io, and Rancher for deployment. I heard that they just replaced Jenkins with Drone.io last semester ( not too sure that they will return to use Jenkins or not) but regardless, I would recommend to get familiar with Dockers, and understand how to use yaml files for deployment process since that is pretty much that you need to modify for deployment.
Oh yeah, pinging @starbeamrainbowlabs and @damfle! Gitea works, but I'm afraid I couldn't figure out how to (easily) test dovecot. As far as I know drone.io needs to use a git host like Gitea for user management (to access files?), so I'm hopeful that that would just work if you just configure Gitea to use OpenID2. Just wanted to let you know!
Well, I can say that Jenkins is a high adaptable dinosaur that never failed on me, and when I say never its because for the last 5 years, I never get into trouble that reading the docs, searching and using the correct plugins or even tracking / opening issues on github could not solve for my work/ home lab. Currently, for my personal projects I have the same builds running on for arm and x86 on gocd, drone.io and travis. But none of then has the same level of flexibility that Jenkins allows me.
​
Also, in the past i have also performed migration from Bitbucket /Gitlab to Jenkins exactly because of the flexibility that it gives to us.
Regarding complexity.... Its just read the docs in fact... I started with bare metal jenkins, then gradually learned regarding deploying on docker swarm, kubernetes using agents fat agents on powerpc / MacOS while it allowed the advantage of having k8s managing the dynamic agents.
​
Jenkins can be heavier or lighter depending on those who maintain it.
We do this on a pretty vanilla Kubernetes setup. Very few teams understand Kubernetes but they all know how to deploy to it. Basically, we allow every engineer to login to Drone (drone.io) and connect a source repo. Then we offer a template to engineers for .drone.yml file. Step 1 is to publish an ECR image, Step 2 is to call Helm - we have a common Helm chart that fits 99% of all use cases (creates ingress, service, deployment, etc.). Happy to go into the nitty gritty if interested.
I would look at using an external usb drive for storing your repos if using a Pi, the sdcard can get corrupted.
while it's nice to write your own servers and feels good when it works, Gogs is a really nice git server that works on the Pi and it has a nice webui plus works with drone.io if you want to do CICD.
Check out Drone :) Or any CI platform should do you well, really! Also if using nginx as a load balancer/reverse proxy when hosting multiple sites off a single VPS (if I'm reading that right?) - I think it would be a better option to use docker itself to host the nginx (or even HAproxy) service(s), then use dockers built-in DNS to perform health checks on yourapp_1, yourapp_2, yourapp_3, etc.
For simple applications, Google Cloud Build works perfectly fine for GKE. You can connect to the k8s and the image registry withouth any configuration, just by adding the proper roles to the IAM user that runs with cloud builder. It uses Docker-in-docker for build steps, similar to Drone.io for example.
Imo is by far the quickest way of setup a CI for GKE, and is really easy to mantain.
Highly recommend drone.io!
Check out my take on deploying it.
Well there's no big magic in using a Jenkinsfile, call some docker-compose commands with appropriate commands and you have similar flow - we've set this up multiple times, there's no magic to it. There's nothing that replicates drone.io's pipelines in Jenkins afaik.
You have to be more specific what your actual problem is when getting it to run - all the instructions are at the Bitbucket site.
Download the executable for your operating system and your jre here: https://drone.io/bitbucket.org/Hottemax/eu4-combatsimulator/files (Note that 64/32 refers to your Java Runtime, not if your OS is 64/32 bit!). No other requirements (double-click should execute the jar, if java is set to auto-start *.jar files.
drone is pretty rad too. their website doesn't do a great job communicating it, but they're also open source and can be run on premises if you're freaky like that.
That didn't really help me to be honest probably because of it being a little outdated but thanks for sharing, Promofaux gave me pretty good instructions how to set up this version of tabbychat 2 properly.
Now I have filters for everything including Skynet, general groupchats, snitches and such
Looked at their repo and if you're using a build of Minigameslib prior to Jan 12 you will experience this.
https://drone.io/github.com/instance01/MinigamesAPI/files
I'd take a look at your timings again after you fix this, though, because there are other concerning things in there.
Can anyone else get in? It logs in for me but then I get booted with a timed out message.
Edit: Could we also update Extra Cells? It doesn't work at all currently and I noticed there is an update for it.
https://drone.io/github.com/M3gaFr3ak/ExtraCells2/files/build/libs/ExtraCells-1.7.10-2.1.20bnull.jar