What is Reddit's opinion of
New Relic?
From 3.5 billion Reddit comments
➔ New Relic website
By popularity on Reddit, this Service is:
39 reviews of this app found across Reddit:
> DAE like to maintain a regexp interface to your regular API?
The UNIX way!
holy shit this site 4realz captures the entire circlejerk lifecycle
- pick stupid name X for your something framework and buy stupid domain name
- sell a bunch of books like "how to use X" and "doing shit we already know how to do in 50 other ways, now in X"
- make a convention and other events about your shit to bring the circlejerk to real life
- pro version. school. training. support. testimonials (but testimonials is not webscale, you need to have a part of your page with a bunch of logos of your clients, like these guys).
The last step is when the product is no longer hip and thus has no reason to exist, but people still rationalize its existence on Stack Overflow etc. e.g, PHP, JQuery, Ruby, Java EE, node.js (too soon?), django.
New Relic is primarily Application Performance Management (APM) software, although like all monitoring tools additional features get bolted on the side like server monitoring and synthetic testing etc.
A lot of vendors offer APM tools as per the Gartner Magic Quadrant:
http://newrelic.com/assets/legacy_landing_pages/gartner_magic_quadrant/chart.png
If you are looking for software that automatically pinpoints slow transactions in your code, or to databases or between services, then App Dynamics is probably your best bet.
For those saying to install Sensu / Nagios / Prometheus / other open source monitoring system.. that isn't APM. It won't do the heavy lifting to present root causes, it will simply present the data. Those tools are for custom monitoring.
For good monitoring coverage you'll need a few tools. I get asked this question all of the time so we even have a page to cover non features so that people know to look at other genres of tool.
https://support.dataloop.io/hc/en-gb/articles/205610495-Non-Features
I'm an architect on a web development team and the question really hits home.
Here's our top 5:
- Ensure Developers are writing Unit Tests
- Ensure QA has a solid set of automated tests, integration, UAT tests
- Create a Continuous Integration Pipeline (CI) and have developers regularly merge back to trunk/mainline.
- Provide a feedback/notification system when a build fails.
- Make releases easy (1 button) and promote the same build across environments.
In terms of the second part. Unit testing and a rich set of automation will be key. In addition, having server side monitoring is important as well. During deploymnets you want to get instant metrics to ensure that errors aren't popping up. New Relic is great for that.
You also would want to consider looking at Canary Deployments and possibly Blue Green Deployments.
Look at this book by Jez Humble. It's a great read for building quality software , including web software.
In addition to this, I would also talk about using Grunt/Gulp to automate all the front end tasks and run tests against the front end ui code.
If you can, get a free trial of new relic and install their PHP extension. It'll start instrumenting/profiling your real user requests and give you information on how long is your app is spending in the database (down to the query level) and or PHP functions. More details here - http://newrelic.com/application-monitoring
Extrahop has impressive solutions. Not sure how far above TCP you care to go, but AppDynamics and New Relic are both great application performance monitoring suites.
For those who don't want to futz around with Nagios and either have small needs or would rather pay a little money: NewRelic is an excellent service for this kind of thing. Install a small daemon, and it monitors/reports on pretty much everything, and alerting/reports are super easy.
We use it at work, and it's been super useful. Also, you can even get a stupid t-shirt for free (really!) if you want.
Nobody has mentioned New Relic? While I haven't deployed them on a massive scale personally it should be able to do what you're looking for. You can start out with a free account as well.
2500 visitors a day isn't really that big. You probably have some inefficient PHP or SQL code that's slowing the site down. Try installing http://newrelic.com/ and using the free trial of the pro version. You can see exactly what part of the app is causing slowness (e.g. HTTP, PHP, SQL) and even drill down into individual DB queries.
> Wow, that's a lot of packages. Are you using both mysql and mongodb? Have you checked so that you're not compiling all resources on each request? >
You should check to make sure that the versions of software you're using are the same. Some package managers are notorious for having out of date Node and Mongo packages.
It might be worthwhile to setup a free New Relic account and add their profiling code to your app.
Then you should be able to narrow down the problem pretty quickly.
I got a free year of learnable premium just by running New Relic on an app, they also have tutsplus if you prefer, although that's less web oriented. Here's a link http://newrelic.com/lp/learnable?utm_source=SITE&utm_medium=promotion&utm_content=popup1&utm_campaign=learnable&mpc=PM-SITE-generic-EN-100-learnable-popup1
Good point, although it is a little bit old school. People interested in quality should have a look on sentry (not python-only) (free) or newrelic
> I basically want to make sure I am logging all of the things I should be so that when things go wrong or even when things are going right
Rookie mistake. Focus on logging whats important (error + stack trace). More logs = more noise. More noise = harder to debug.
- Log to different files.
- Log to different files.
- Errors. If, and only if, there is a specific error that keeps occurring and stack traces are not enough, consider logging everything until you catch the error.
- Use console.log or something like https://www.npmjs.com/package/debug. Avoid loggers that allow logging complex structures. See #5.
- It might seem awesome at first ("easy" to filter, capturing all the data at the time of the log, etc.) and it can be useful, however, it makes logging system, logging aggregation and logging analyses process a lot harder. Focus on error logging.
- Think what is the error. What would cause the error. And what would help to solve the error. In general, stack trace is your primary tool. After that, scope and environment values at the time of the error might be useful.
- 1 error entry is more useful than 1M debug logs.
- If you are a one-man band, I recommend http://newrelic.com/ or either of their competitors. Thats specifically for measuring/ monitoring latency of external services. It makes it super easy.
Using tuned-adm is a must for me in all environments. I've customized profiles to make sure "virtual-guest" and "enterprise-storage" cover my use cases; also deployed contextually via Puppet. Basic stuff like removing write barriers and getting noop or deadline schedulers on my block devices. That tends to benefit VMs and bare-metal alike.
While I don't Datadog, I definitely NewRelic - Server monitoring is basically free for 24-hours - I pay $150/mo. for unlimited hosts and 3 month's retention. Any major differences you see between them?
On all of our projects at work, we use NewRelic, Monolog, and Sentry (Raven). We don't log everything, but we do log all the steps in our complex flows.
Sounds like a devops-type tool is the thing you are looking for.
Install New Relic. You will start with a pro account free for 30 days, that will give you lots of useful info from both the server and the php software.
A more complex way: Use Xdebug and something like https://github.com/jokkedk/webgrind. This will give you more specific info about the more resource consuming parts of the code. You should not use Xdebug in a production/live site though.
Also, are you using cache plugin in Wordpress?
The free version of NewRelic (http://newrelic.com/) might be what you are looking for.
It has some limitations in the free version, but you can definitely set up your alerts to notify you on a per server or a per vhost basis.
I've posted this before in another thread but Tuts + offer a free year to anyone who signs up to a new relic trial - save yourself some cash. (Same with Lumosity which is fully web-oriented) Here is a learnable link, I'm sure you can find a tuts+ referral on google http://newrelic.com/lp/learnable?utm_source=SITE&utm_medium=promotion&utm_content=popup1&utm_campaign=learnable&mpc=PM-SITE-generic-EN-100-learnable-popup1
If you are developing on a Mac, watch out for environmental differences in your gems/setup if you deploy to Linux.
If you have background jobs (resque, delayed_job, etc.) or internal server monitoring (god, monit, etc.), make sure your deploy takes care of restarting those if necessary or updating their configs when they change in your repo as part of deploying.
You might want to have automated post-deploy tests in your capistrano script (you're using capistrano?) to make sure all the things are working post-deploy. Just add some relevant things in a test/deploy and invoke them however you want in an after:deploy step. Bonus points if you do during the deploy so you can roll back. Haven't gone to that level of trouble myself, but it would be neat. :)
Bad migrations - indexing lots of rows, actually broken code, etc. - will bust your balls on high traffic (high no. of records?) sites. Make sure you have a way to get real (or very close to it) data to your local system and test everything before deploys. Maybe a rake task like “rake get_and_import_production_data”.
Scout is popular for monitoring, as is New Relic. They both have their upsides and downsides.
Good luck!
I use NewRelic (http://newrelic.com/).
It will not only monitor your website and notify you if it is down (via emails, text messages, or Slack integrations - check out NewRelic's "Synthetics" feature) but also can measure website performance, server load and other interesting data.
I've been using its free tier for several projects and I find it very useful, reliable and easy to set up.
I use Monolog + handlers depending on the context.
For production environment these handlers include Newrelic, and newrelic in turn might be configured to send me alerts for more than X app errors in a timeframe as well as other conditions such as CPU usage, free space on HDD, etc. The alerts might come through email, hipchat, slack and others.
For dev environment I'm using PHP-Console handler, which sends errors and debug messages to the browser's console.
Every environment also has a Streamhandler appending data to the application's php error log. This way I have one place to check for application errors as well as errors from php-fpm. Some times, like when you forget to update your composer dependencies, the error will happen before the app has time to instance Monolog, so no matter what handler you use, you'll end up checking for the php error log sooner or later. You can have this log checked in an online dashboard like the ones mentioned in these comments. I'm also used LogEntries for this purpose.
Oh right. I completely forgot that! I will edit that in! Thanks!
Still. It's a Flash framework that doesn't even come with support for TCP, binary sockets, file systems or a local database (e.g. SQLite) meaning quite a bit of work on top of it (see the heavy dependence on Internet Explorer). In addition Flash was always primarily for visuals with it's strong points on vector graphics and animations. Granted they are pretty much THE reason why we have all the cool login screens (which are flash animations) but besides that if you look at it... it's a very poor choice of framework for a launcher. Especially regarding long term support.
The main reason I see why they stick to it this consistently is because of the monitoring tools they have. Riot is using the NewRelic monitoring tools to monitor their website performance but also user behaviors in the shop and similar things (this is not facts but that tool is very much capable of. When looking at the cookies set by LoL I found that they are using this service so the assumption is close that they would use it for that purpose) which would not be possible to be properly integrated into a native application so they would still rely on websites.
This combined with the fact that they very clearly want the login screens (for whatever reason) which would require them to change up their workflow massively or still fall back on flash looks like solid reasons why they stick with it... even though it still is a poor choice imo.
I've also just started using New Relic (free tier) and it gives you a breakdown of what causes the loadtimes on your server, which gives some insight. I'm currently looking to see how easy it is to get a WP site working with it so I can let you know if you want?
Looks like they've expanded to a lot of offerings.
This is server monitoring (whether it's bare metal or VPS). Their agent sends CPU/network/disk/etc info on a continuous basis to New Relic, which you view via their dashboard.
Honestly, look at NewRelic Servers: http://newrelic.com/server-monitoring
It's free, easy to install on a *NIX server, and will collect metric data as well as alert you when things start using too much CPU, host goes down, etc. This way you don't need to rely on a monitoring solution that runs on the same hardware as the stuff you're monitoring. I mention this because you say you have your own cloud server but don't mention any other boxes.
As for monitoring logs in a terminal, just learn to love tail/tailf. If you want to follow your apache log in the terminal just do a:
tail -f /var/log/apache2/access.log
Replacing the log name with the actual log you want to monitor.
A combination of CollectD and graphite can do some basic monitoring and graphing. Maybe with some custom web interface based on d3.js you can get closer to what you're looking for.
We've used Nagios for years when I was in a consulting company. If you have a dedicated team to maintain it, you can manage, although I don't think it is worth it any more. The cost of setting up and maintaining your own solution, vs the cost and features that you get from a monitoring SaaS outweights the positives.
If you don't want proprietary agents on your servers, or want to have some power over your UI, I can suggest our own Mist.io: https://mist.io You can install the UI (it talks to the service backend to get monitoring data) localy too: https://github.com/mistio/mist.io
Otherwise there are several SaaS options, most notably New Relic: http://newrelic.com/
- put in place some instrumentation to measure your application, end-to-end. you won't even know what to scale unless you're measuring. I recommend New Relic for this kind of thing.
- identify your bottlenecks, based on the results of your performance metrics. the most likely spots for a bottleneck to appear (in my experience anyway) are load on your database, load on your app servers, or network bandwidth bottlenecks.
- solve the bottleneck you've identified. each one requires detailed analysis of your tech stack and code base, so I can't really give a general guide here. in general though databases tend to benefit from vertical scaling (run it on better hardware). app servers benefit from both horizontal and vertical scaling. network bandwidth problems usually requires a better host, and probably using a proxy server and load balancer in front of a pool of workers.
New Relic offers a server monitoring service. I've got plenty of graphs for things like I/O and memory usage. I can also set up alerts if anything gets too crazy. Free for as many servers as you need.
I'm not quite clear on what your goals for this project actually are, but it sounds like you're just trying to do some data aggregation for statistics or analytics. There are a lot of SaaS services out there already built to do such a thing. If you don't have a lot of time to dedicate on building one, just use one of those and be done with it.
Here are a couple that are well-known, and do slightly different things depending on your needs:
Sounds like a good use for XMPP
As for server monitoring, I think using a 3rd party is always a good idea. You can look around, there should be some out there that offer basic free monitoring, New Relic is one example I believe.
Crash monitoring can be scripted on the host itself. For example you could setup a cron job that checks to ensure the server is running every few mins and restart it if not. If you hot on a Linux server that is :)
We use New Relic (http://newrelic.com/). We're still on the free tier, and it's already incredibly useful. Nice mobile app too. The few days after we deployed these changes I was checking the app incessantly.
If you have good developers who can interpret the data, new relic is worth the $. http://newrelic.com/
Sign up for a trial and you get the full thing for a month, will probably reveal some useful info.
Have you run the apc.php to check your cache is sufficiently large? Are you using memcache?
Specifically for web application monitoring, New Relic's offering is pretty incomparable. http://newrelic.com
Sensu is a newer monitoring framework. http://sensuapp.org/ Cloud friendly architecture, rest api, supports Nagios check scripts.
But really - try icinga2
Late again this month, sorry: New Relic continues hire people at quite a clip. I work here in a mid-level job and absolutely love it.
http://newrelic.com/about/careers
Note: I'm very sorry if you got in touch with me about a job and I never responded. Please ping me again and I'll write back.
Thanks for your reply! I talked to a couple other engineers I know about this. One used papertrail for his servers, the other uses the same logstash solution you mention. I ended up going with logentries because it was simple to set up and free for now. Logentries also has alerting which is nice.
As far as timing/performance data goes, have you ever looked into using New Relic? While you can use logs for this, I've been told by other engineers to separate logging from performance monitoring. I've been considering giving New Relic a try... looks pretty powerful and easy to setup.
Tuts plus is giving one year of premium free if you install newrelic for monitoring an app
We've been using newrelic for a bunch of our servers, and it has been really great. You can start a trial account, and in 2 weeks it turns into the free package.
You can install a daemon that monitors just PHP, but they've recently added server monitoring. I'd recommend both, to get to the root of your problem (both are free)