For anyone curious, this was done with a combination of ds18b20 sensors, an HA7Net controller, the home's existing phone wiring (plus a few special cable runs), and a linux box running cacti. The usual response to my showing this to people is a quizzical look followed by a comment about my apparent surplus of free time.
edit: by request, more details and some photos here.
These are some ideas (sadly I don't have the chance to write much details.)
None if these projects involve much (or any) programming, at most it may be a simple script or cron job to add.
Have you been reading the Cacti doc? http://www.cacti.net/downloads/docs/html/
It's very good.
The graph, when created, should have green for incoming data, and blue for outgoing data. Remember that this is from the perspective of the port, so on a switch, incoming data (green) means output from the server, while outgoing data (blue) means input to the server.
Make sense?
You can do SNMP polling with a free tool like cacti based on in/out bytes. It'll do 95th percentile and all that too.
Edit: Cacti also makes it easy to do user management so you could give the client web login to their graphs to check whenever they want, and scale this out to multiple customers.
Sorry to be a pedant but I think you mean SNMP;
SNMP - Simple Network Monitoring Protocol,
SMTP - Simple Mail Transfer Protocol
If you can access the gateway, try and see if there is an option or tick box to enable SNMP (you'll need to set up a community string which is basically used like a password to log in and collect SNMP data)
If you can get SNMP working on your gateway/firewall/router you can download a program like cacti and run that on your PC. Cacti can collect the SNMP data from your router and turn it in to graphs and stuff so you can monitor the network utilisation on each port of the router/hub and see which port/host is using all the bandwidth.
Of course if you can log into your router, just checking the interface statistics manually might be simpler.
Edit: Nagios is good for this too but I think it only runs in Linux, cacti will run on a windows PC.
Oh, and I forgot. I have stood up weathermaps quite a few times. On some recent projects, I stood up Cacti, and then the network weathermap.
I have Cacti export everything to a straight html directory, and serve that up to my client. It displays on a nice screen for them. The weathermap took some work (getting everything arranged), but I ended up with something that matches their physical campus with wireless links. The links will change color based on the amount of traffic going through them.
If you just want the WAN port utilization and session numbers, cacti can graph this. Or its old-school predecessor, MRTG.
Really you just need anything that can graph an SNMP OID.
You might find after installing Cacti that you find lots of other things to graph. I particularly like the network weathermap plugin.
The problem people are pointing out is that 99% of users are fine with 10/100 and CAT5. To start ripping shit apart without knowing where the bottlenecks are is going to be a waste of time/money and make you look bad.
People are trying to point this out and you're lashing out at them as if they're clueless. Maybe you've actually done some troubleshooting and have metrics to support your plan but it certainly isn't detailed or even implied in your post.
1) Users do not need gigabit connections -- a simple stress test will determine this if you run one
2) CAT perfectly fine for this -- check your switch/router logs and port counters for errors
3) Switches and Routers need gigabit between them -- you can upgrade the hardware or buy module. Model numbers would be helpful to point you in the right direction.
4) It doesn't appear that you've done any bench-marking of your network or servers
I've seen stuff like a Spiceworks inventory scan grinding the network to a half before...Hopefully, you already have something in place for monitoring the network traffic but I'm guessing you don't.
If you have the time, throw together a cacti server and put the network hardware in there and try to see what's going on when things get slow.
If 2500 pings can strain your network, you have bigger problems. If you set it to run a full test every hour, you're looking at one ping every 1.44 seconds. That's hardly taxing.
Signal strength, number of clients per AP, and number and channel of nearby equipment are going to be important in determining expected link quality. Those should all be easily queryable, either through SNMP or a vendor-specific API. Smokeping defaults to a ping probe, but can be expanded to query just about anything that can be distilled to a number. It wouldn't be hard to have that track many of these, though other projects such as Cacti might be better suited.
Cacti. http://www.cacti.net/ Far easier than Nagios, IMHO. Plus plugins like Network-Weathermap http://www.network-weathermap.com/ really make it stand out.
Sample Weathermap: http://www.network-weathermap.com/files/images/weathermap_13.preview.png
I use our installation for alerting on server disk space, link up/down on important nics, cpu graphing, user login graphing. One of our weathermaps has information on nearly every facet of our AD infrastructure, from DNS response times, to DHCP lease numbers, to CPU performance on the DCs. Helpdesk loves it for at-a-glance troubleshooting.
> The problem with New Relic is that it will cost us a lot to have it set up for all of our clients. We can definitely offload the costs to our clients but then they will moan about the increase in pricing for our contracts. That's why I was hoping maybe there were cheaper alternatives. I will take a look at Nagios but judging from their products, they are also quite expensive. Any advice here?
Nagios is open source and free. But you do need a server to host it on (or even a virtual machine would do).
I use newrelic, and it can get expensive real quick. Its pretty good but I'm not sure I'll renew next year as it doesn't offer that much more than my free tools already do (although newrelic is prettier and way more user friendly).
Look into cacti also, for long term resource monitoring and capacity planning. http://www.cacti.net/
I'd start with server monitoring if I were you. No point having automated build and depoyment if your servers are going down regularly
Sounds like to me he wants to do maybe some SNMP monitoring for thing like system utilization etc. If that is the case check out http://www.cacti.net and http://forums.cacti.net/viewtopic.php?f=12&t=44560 for specifics on integrating PAN into cacti.
Look at cacti, it's a well featured snmp monitoring app that just uses the normal LAMP stack. it's light weight enough to run well on a Pi that's being used for other things. You'll probably want to use the C version of their poller (spine) over the php-based poller.
You could consider using another single board computer that supports linux, such as the BannanPi or O-Droid. They will run SNMP and could support 3/4g with a dongle. For graphing and logging you could consider Cacti an open source RRD based grapher. Then build some remote access either open vpn or alternative to gain access via the 3/4g IP address. Cacti can build graphs on SNMP and pings etc Cacti can also send email alerts when a resource goes down or threshold is reached. If this is for a network you could hang it off of a switch port configured for monitor or R-SPAN and capture traffic via TCPdump for use in wireshark. edit: fixed some wording added info about pings.
Take some edX courses - Here's a great intro to Linux: (FREE AND ONLINE! WOO!) https://courses.edx.org/courses/LinuxFoundationX/LFS101x/2T2014/info
Set a goal and tinker with it until you get it to work and understand it as much as physically possible. Since monitoring seems like a problem for your setup, try putting your newly gained Linux knowledge to use. Start by creating a (virtual) server and install some of the open source monitoring software.
At my workplace we use nagios: http://www.nagios.org/
and cacti: http://www.cacti.net/
Read certification books. (A+, Network+, Linux+, Security+, etc) -O'Reilly offers deals quite frequently on the electronic copies of their material, which can be quite affordable.
Learn a scripting language - ruby, python, powershell commands, etc.
Stay curious! Set up a lab at home and think of something cool to set up and get inspired to want to get it to work perfectly. Then break everything and figure out how to fix it.
edit: formatting
I'd say make one of them a Linux box and install Cacti for network monitoring. Once you get everything hooked up and configured correctly you'll be able to comb through your network's statistics with a fine-tooth comb.
It depends on the OS you use on them.
One thing you basically cannot do is run all three of these together like one big machine. Clustering technology exists, but it is not suitable for desktop use and it's very (very very) complex to setup right anyway.
If you want to offload some tasks to a specific machine then this often depends on the software used, some (very expensive) software has build in clustering capability to create "render farms".
In Windows you can use PsExec to start a program on another machine.
For monitoring machines you can use software like Cacti.
You can also use remote desktop (Windows Pro and higher editions), or some form of VNC to centrally control your machines.
Yes, with the device in it's default configuration, you can plug things in the way you've suggested. If the config, or vlan.db file have previously used data in them, then some ports may be unable to communicate with others.
The ports labeled "molex" take small transceivers, in this case know as GBICs. This is an example of one that will fit in your switch and take a copper cable.
For monitoring traffic, you'll probably want to use a protocol called SNMP, and some software running on your [game] server that will graph this data out. I like Cacti a lot, but it may be a bit involved for just a LAN party. You're correct, this SNMP data will travel via TCP/IP over your ethernet ports on the switch, vs the console port which just provides access to the device for management.
Yes that's typical. If I was paying based on our 9-5 usage I would be paying for about a 10 meg connection as well.
As a side note, it's pretty easy to setup Cacti to monitor your firewall and find out exactly what your usage is. They even have templates that will draw your 95th line on the graph as shown in the Wikipedia link.
Nice idea, I think Cacti does something similar with storing device configurations.
Edit* You may also want to look into moving the "Get Started" box down as the bird network thing behind the words makes it difficult to read.
The primary one is not a BB app - it's Cacti
Any of the BB apps I have worked on are far lower then your stats - but they are internal. That's my excuse :)
Didn't mean to come off harsh on my side.
you can use iperf3 to run speed tests internal to your network. Install it on your computer, and a local server of some kind. When someone experiences slowdown - perform a test and see what you're bandwidth is. That will let you know if the slowdown is on the LAN.
Past that, on your WAN devices, see if you can use some bandwidth monitoring to see if you're over utilizing your circuit. Most equipment can be setup for something called NetFlow (or N-Flow or J-Flow or whatever-Flow) that keeps track of bandwidth on a particular interface. Then you can use something like Cacti to poll this information and show it to you in graph form.
Hope this helps.
As others have said, Wireshark isn't really your best option here. Some have suggested Netflow/sFlow, which will certainly work, but another option is a traffic grapher such as Cacti or MRTG, which will poll your switch every five minutes and get bandwidth statistics, then display them in an easy-to-read graph.
It uses RRDTool to render graphs, so yeah.
But, is uses RRDTool, so you can take the data from MySQL (PDF) and do what you wish with it.
With a limited budget have a look at http://www.cacti.net/ Cacti is open source and I know guys monitor literally thousands of systems with it. Maybe it's right for you maybe not worth having a look though.
I like python for that. Also BASH scripting can do the job with the net-snmp commands like snmpwalk an snmpget.
In python there is pySNMP but it's hard to handle. I like hnmp (https://github.com/trehn/hnmp) It's more pythonic, more simple with good results.
Cacti is another great tool to graph the bandwidth and all kind of stuff. If you can get a number then you can graph it. http://www.cacti.net/
Edit: my terrible grammar
Use Cacti?
When you run the commands on the devices, it only shows the current stats. If you load cacti on a server it will poll over time so you can see how it changes during the day.
Checkout Cacti at: http://www.cacti.net/what_is_cacti.php
It periodically gathers ping and other statistics (if your network hardware supports SNMP). Data is logged to a Round Robin Database (RRD) and displayed on a web page. That page would not display well on 16x2, but you could have your python just pull data from the database to display. Runs well on the Pi2.
If your router supports SNMP, there are linux command line tools you can call to pull information from it in formats much easier to parse than fields from the router GUI. Search for SNMP, RRDTool, and MRTG for other tools that might help you.
Indeed.
Observium is a lot more than a solution to monitor VMWare, but it is a very smooth way to do it.
If you like to get your hands something more targeted, I would suggest Cacti but the learning-threshold is quite high.
We used to use cacti for network perfromance monitoring. I'm not sure if it will do cpu and memory stats though, we only used if for graphing interface utilisation. it's open source and free!
Now we are paying for a solar winds license which does all the things you mentioned but it isn't free.
iirc, cacti can monitor individual processes. If not, it can at least give you a good idea of load over a period of time (with pretty graphs, etc.) I would install it and make the web interface available remotely (port forwarding, vpn, ssh tunnel, etc.)
If that's too much then I agree with others. Use top\htop.
Requires a bit of tech knowledge, but if your router is SNMP capable, you can set up cacti and graph the traffic per port
Edit: Or check out Network Probe
> seeing which sites are always maxing out their cpu etc
http://www.nagios.org/ plus http://drupal.org/project/nagios
Nagios is a monitoring system that tracks a system's conditions and can send triggered alerts. The Drupal Nagios plugin allows for monitoring:
This is in addition to out of the box monitoring of things like service health (MySQL, Apache, memcached, etc.).
If you want to get into system monitoring and graphing, you can look into http://www.cacti.net/ as a RRDTool tracking solution. It is by no means the only solution, but this is a starting point.
Is that more aligned with what you were looking for?
So Toby built MRTG and it was celebrated because it queried and graphed things...but it has major drawbacks...
First, it graphs after it queries, not on demand. This means that every graph is rendered, whether it's looked at or not. This is really wasteful.
Second, there can only be two variables, which is ok if all you want to look at is "in" and "out". If you want to look at, say, VLAN traffic, then you need one graph per VLAN (and remember, each graph is ALWAYS rendered, whether its needed or not)
Third, there's no ability to graph anything lower than zero. That precludes a TON of things, so you end up making more graphs for when you need to plot things on the negative side.
It's definitely not great.
Look at tools that use RRDtool, which is what Toby wrote to replace MRTG. Cacti is kind of the defacto go-to for RRDtool management.
Graphite is another great graphing solution that has far, far more flexibility than MRTG.
What kinds of things are you monitoring? Do you have up/down notifications set up through Nagios or some other system status monitor? It may be possible to tie things together.
So SNMP lets you get periodic data for various counters (OIDs). The OIDs are grouped into things called "MIBs". The MIBs can be called tables. In this case, the ifTable is a collection of counters around your interfaces. You are interested in ifInOctets and ifOutOctets. link to cisco site on this.
Implementation can be on cacti, which is kinda awesome, but probably overkill.
If you don't have dd-wrt, but want stats on your whole network and you have windows or unix or know how to port it to mac, then cacti is awesome too. This one is good too, but doesn't do logging, etc - just shows what is going on right now. All of these work off of SNMP, which the AEBS, most decent routers support. If anyone knows how to compile/port/whatever is needed for Cacti to run on a mac, hook me up... as I don't have a clue, but would like it.